Job Description & Details
This is a heavy-hitting senior engineering role for someone who has moved past standard CRUD apps and is actively building production infrastructure where AI agents are first-class citizens. You aren't just calling OpenAI APIs here; you're architecting resilient systems that handle non-deterministic outputs, rate limits, and token economics at scale using Java and modern cloud tech.
What You'll Actually Be Doing
You'll spend your days designing, building, and scaling high-throughput microservices and messaging pipelines that support heavy AI workloads. Expect to tackle messy production challenges like managing model behavior under load, figuring out caching strategies for LLM responses, and balancing low-latency requirements against cloud infrastructure costs. You'll be expected to dogfood AI-assisted development tools daily, shipping fast while maintaining clean microservice boundaries across Kafka, Kubernetes, and various SQL/NoSQL datastores.
The Core Tech Stack
Your daily bread and butter will be Java and Spring Boot running on Kubernetes, interacting with event-driven architectures via Kafka or ActiveMQ. On the data side, you need deep fluency in both relational databases like MySQL and distributed NoSQL systems like DynamoDB or Cassandra. Crucially, your superpower here needs to be hands-on experience integrating LLM APIs and agentic frameworks, paired with front-end competency in either React or Angular to tie user interactions into these AI-driven backends.
Interview Expectations
Expect the hiring panel to test your real-world system design instincts by asking you to whiteboard a scalable architecture that orchestrates multiple LLM agents while gracefully handling API rate limits and token-cost spikes. When they ask about failure modes in non-deterministic systems, they want to hear concrete war stories about how you detected silent model regressions or handled cascading timeouts in a microservices environment, rather than textbook definitions.
Application Advice
Your resume needs to skip the generic buzzwords and immediately highlight shipped products where AI was core to the architecture, not just a tacked-on feature. Make sure terms like Spring Boot, Kafka, Kubernetes, and specific LLM orchestration patterns or agentic frameworks are prominently featured in your experience bullets to clear the initial ATS screening.