Job Description & Details
This is a heavy-duty data migration gig based out of Houston on a hybrid schedule, and it is not for the faint of heart. You'll be right in the trenches moving massive volumes of enterprise data, dealing with messy source systems, and making sure the business doesn't hit a wall during cutover.
What You'll Actually Be Doing
Your day-to-day will revolve around untangling legacy data structures and making sure everything lands cleanly in the new target architecture. You will spend a lot of time writing source-to-target mappings, profiling messy data, hunting down data quality bugs, and working closely with engineering and architecture teams. When migration weekends roll around, you'll be knee-deep in validation, reconciling record counts, and making sure transformation logic didn't drop critical keys.
The Core Tech Stack
You absolutely must know your way around Python and PySpark, and you need practical, hands-on experience running workloads on Databricks paired with either Azure or AWS. They aren't looking for someone who just reads documentation; you need to have built end-to-end ETL/ELT pipelines in production environments where scale actually matters. If you can't talk deeply about distributed data processing and cloud data warehouses, you'll struggle here.
Interview Expectations
Expect the hiring team to grill you on how you handle data reconciliation failures during a high-pressure cutover weekend, so be ready with a war story about finding a discrepancy between source and target at 2 AM and how you fixed it. They are also going to test your PySpark optimization skills—specifically how you handle shuffle bottlenecks and memory tuning on Databricks—because they want to know you won't write code that brings the cluster down.
Application Advice
Make sure your resume doesn't just list keywords; explicitly highlight your end-to-end ETL implementations using Python, PySpark, and Databricks in an Azure or AWS ecosystem. Ensure terms like 'source-to-target mapping', 'data cleansing', and 'reconciliation' are prominent in your project bullets so you easily clear the ATS screening for this role.