Back to Jobs

Senior AI & Data Pipeline Engineer

Not Disclosed

Job Description & Details

This is a serious platform-building role where you will be designing enterprise-grade multi-agent AI frameworks and data pipelines from scratch. If you enjoy wrestling with complex agent-to-agent communication patterns and making LLMs behave deterministically in production, this is the exact kind of problem space you want to be in. It is high ownership, requiring you to bridge traditional data engineering with cutting-edge agentic workflows.

What You'll Actually Be Doing

Your days will be split between architecting robust data flows using PostgreSQL and Snowflake and building out the orchestration layers for autonomous agents. You will handle everything from writing optimized SQL queries and designing schemas for downstream analytics to tuning prompts, setting up RAG workflows, and ensuring your LLM pipelines have proper observability and security. Expect to deal with messy, ambiguous business requirements and turn them into clean, highly scalable Azure-native production systems without a lot of hand-holding.

The Core Tech Stack

You need deep, production-level fluency in PostgreSQL Server and Snowflake because the AI outputs are only as good as the data pipelines feeding them. On the AI side, heavy hands-on experience building multi-agent architectures on Azure is non-negotiable, alongside a solid grasp of the full LLM lifecycle including prompt engineering, evaluation, and observability. If you haven't deployed reliable, scalable agent-to-agent communication patterns in a live enterprise environment before, you will struggle here.

Interview Expectations

Expect the hiring team to grill you on failure modes in multi-agent orchestration, such as how you handle infinite loops, conflicting agent outputs, or state management during transient network failures. They are secretly looking for battle-tested scars—they want to know you have actually debugged a distributed agent system in production rather than just following a tutorial. You will also likely have to defend your schema design choices in PostgreSQL and Snowflake when scaling up to millions of rows for RAG context windows.

Application Advice

Skip the generic AI buzzwords and make sure your resume explicitly highlights production deployments of multi-agent frameworks, Azure integrations, and heavy SQL optimization work. Use exact keywords from the prompt like "multi-agent framework," "Snowflake," "PostgreSQL Server," and "Azure-native" to easily clear the ATS, but back them up with metric-driven bullet points detailing the scale and impact of the systems you've built.