Back to Jobs

Data & AI Solution Architect

Not Disclosed

Job Description & Details

This is a senior‑level Data & AI Solution Architect role focused on building end‑to‑end analytics platforms for finance teams, with a heavy emphasis on Databricks and cloud services. You’ll be in a hybrid office split between Boston or Jersey City, so you need to be comfortable showing up three days a week.

What You'll Actually Be Doing

You’ll design and ship data pipelines that pull raw transactional feeds into a Lakehouse on Databricks, then expose curated tables to downstream reporting and ML models. Expect to spend a good chunk of time translating finance‑centric use cases (risk, forecasting, regulatory reporting) into scalable Spark jobs, while also shepherding data governance, security, and cost‑optimization policies across AWS or Azure. Collaboration is key – you’ll be the bridge between data engineers, product owners, and finance analysts, constantly iterating on schema designs and performance tweaks.

The Core Tech Stack

The non‑negotiables are Databricks (Spark SQL, Delta Lake), a major cloud provider (AWS or Azure), and strong data‑modeling chops in relational/OLAP contexts. You’ll also need to be fluent in Python or Scala for pipeline code, and comfortable with CI/CD tooling (Terraform, GitLab CI) to keep the infrastructure reproducible. The finance angle means you’ll be dealing with strict compliance (e.g., SOX) and need a solid grasp of data lineage and masking techniques.

Interview Expectations

  1. “How would you optimise a Spark job that’s spilling to disk when processing a 10 TB finance dataset?” – They want to see your understanding of partitioning, caching, broadcast joins, and cluster sizing, plus cost‑aware tuning.
  2. “Walk me through designing a data‑governance framework for a multi‑region Databricks deployment handling PII.” – Expect them to probe your knowledge of Unity Catalog, role‑based access, encryption at rest, and audit logging.

Application Advice

Tailor your resume to shout out the exact buzzwords the posting uses: “Data & AI Solution Architect”, “Databricks”, “Hybrid”, “Finance”, “AWS/Azure”, “Spark”, “Delta Lake”, “SOX compliance”, and “CI/CD”. Highlight any end‑to‑end projects where you built a Lakehouse for finance or risk analytics, and quantify impact (e.g., reduced reporting latency by 40%). A short cover note that mentions the Boston/Jersey City hybrid model and your willingness to be onsite three days will help you pass the ATS and catch the recruiter’s eye.