Job Description & Details
This is a heavy-duty platform and observability role focused on wrangling massive distributed systems, log analytics, and tracing pipelines at enterprise scale. If you enjoy building rock-solid monitoring infrastructure rather than just putting out fires, this gig gives you the ownership to architect tools that entire engineering organizations rely on.
What You'll Actually Be Doing
Your day-to-day will revolve around designing, deploying, and operating massive observability platforms. You will spend a lot of time scaling Splunk clusters, managing Elasticsearch for high-volume log analytics, and building out distributed tracing pipelines with Grafana Tempo and OpenTelemetry. Expect to write plenty of Terraform to automate infrastructure and collaborate with various teams to ensure they are instrumenting their services correctly without crushing system performance.
The Core Tech Stack
You are going to need deep, hands-on experience with Splunk Enterprise or Splunk Cloud, including managing indexers and search head clusters. Alongside Splunk, you must be comfortable running large-scale Elasticsearch clusters and scaling Prometheus, Grafana, and Kafka. Because this role touches distributed tracing, you need a strong grasp of OpenTelemetry and Grafana Tempo. Infrastructure as Code via Terraform and scripting in Python, Go, or Bash will be your daily tools to keep this ecosystem running smoothly.
Interview Expectations
Expect the hiring team to grill you on failure modes in distributed logging and tracing. They will likely ask how you handle backpressure and ingestion bottlenecks in a high-throughput Kafka and Elasticsearch pipeline. The interviewer is secretly trying to figure out if you understand the actual performance trade-offs of storing high-cardinality metrics and traces versus the raw storage costs, so be ready to talk numbers and architectural decisions.
Application Advice
To get past the ATS, make sure your resume explicitly highlights your hands-on experience scaling Splunk, Elasticsearch, and OpenTelemetry pipelines rather than just listing them as generic skills. Emphasize your Infrastructure as Code background using Terraform and your ability to write automation scripts in Python or Bash. Frame your past achievements around reliability, scale, and reducing MTTR to immediately catch the hiring manager's eye.