Job Description & Details
This is a heavy-hitting data role that sits right at the intersection of traditional data engineering and modern generative AI. If you like building scalable pipelines while experimenting with LLMs and RAG architectures in AWS, this gig actually delivers on the hype without drowning you in red tape.
What You'll Actually Be Doing
Expect to spend your days knee-deep in data pipelines, figuring out how to ingest massive datasets and wire them up with AI services like Amazon Bedrock. You will translate vague product requests into concrete data models, build out robust REST API integrations, and ensure your PySpark jobs don't crash when processing heavy production traffic. Communication is huge here, meaning you'll need to explain complex architectural decisions to non-technical stakeholders without sounding like a textbook.
The Core Tech Stack
You absolutely must know Python and PySpark inside and out for crunching data and building automation scripts. AWS is the cloud playground here—think Lambda, Glue, and Step Functions—so serverless architectures need to be second nature to you. Toss in SQL for data wrangling, REST APIs for system connectivity, and hands-on Generative AI chops involving RAG and AI agents, and you have the exact toolkit needed to survive and thrive in this codebase.
Interview Expectations
Be ready for the hiring manager to throw a messy scenario at you involving a slow-running PySpark job processing gigabytes of unstructured data, where they'll want to see how you optimize bottlenecks and partition your data effectively. They are secretly testing whether you actually understand distributed computing fundamentals or if you just copy-paste snippets from Stack Overflow. Secondly, expect a deep dive into your AI architecture experience; they'll likely ask you to whiteboard a secure RAG pipeline using Amazon Bedrock, evaluating whether you know how to handle latency, context windows, and enterprise data privacy constraints.
Application Advice
Skip the generic template and make sure your resume explicitly calls out your hands-on experience with PySpark, AWS serverless components, and specific GenAI frameworks like Bedrock or LangChain. The ATS here is definitely tuned to flag exact matches for Python, PySpark, REST API, AWS, and SQL, so weave those keywords naturally into your work history bullets rather than hiding them in a boring skills matrix. Emphasize any past projects where you owned the architecture end-to-end, as autonomy is a massive selling point for this team.