Design ML Platform

Expert60 min
1 / 30
understanding•7 min read

Problem Framing for ML Platform

How Problem Framing for ML Platform (understanding) informs ML Platform architecture and interviewer depth.

What we are building

An ML platform is the internal product that lets hundreds of data science and applied ML teams train, evaluate, register, and serve models without reinventing orchestration, metadata, GPU scheduling, or governance on every project. Think Vertex AI, SageMaker Studio, or Databricks ML: not a single model, but the factory that produces and operates models safely.

Primary personas

  • ML engineer owns pipelines, feature contracts, and deployment automation.
  • Data scientist runs experiments, compares metrics, and requests promotion.
  • Platform SRE guarantees quota fairness, cluster health, and incident response.
  • Risk/compliance audits lineage, access, and model documentation.

Core journeys

  1. Ingest labeled datasets with schema validation and access controls.
  2. Launch distributed training with reproducible code+data+config snapshots.
  3. Track experiments automatically (metrics, params, artifacts).
  4. Register model versions and deploy to batch or online endpoints.
  5. Monitor drift, latency, and cost with automated rollback hooks.

Why interviewers ask this at Google/AWS/Databricks caliber

They want evidence you can separate control plane (metadata, policy, scheduling API) from data plane (object storage, feature serving, inference). They also probe whether you understand reproducibility and governance—not only Kubernetes jargon.

How to open this interview

Lead with the factory framing — not a model, but the system that produces and operates models — then immediately split the control plane (metadata, policy, scheduling API) from the data plane (object storage, feature serving, inference). State the invariant you will protect throughout: reproducibility (code + data + config snapshots) and lineage, because that is what separates a platform from a pile of notebooks.

Key Highlights

  • •An ML platform is the internal factory (Vertex AI / SageMaker / Databricks) that lets many teams train, register, and serve models without rebuilding orchestration each time.
  • •The defining architectural split is control plane (metadata, policy, scheduling API) versus data plane (object storage, feature serving, inference).
  • •Four personas set the requirements — ML engineer, data scientist, platform SRE, risk/compliance — each demanding a different guarantee.
  • •Reproducibility is the core invariant: every run snapshots code + data + config so any result can be regenerated exactly.
  • •Measure success in platform terms — GPU utilization, queue wait, time-to-rollback — not model accuracy alone.
  • •Governance and lineage are first-class: the platform must audit who trained what, on which data, and how it was promoted.
Interview signal
State the invariant first (reproducibility, lineage, or latency), then mechanism, then how you validate in staging with shadow traffic.
Avoid this
Treating notebooks as the production interface—platform value is governed pipelines, versioned artifacts, and auditable promotion gates.

Section Rescue Kit

Buzzwords to use:

ML Metadata ServicePoint-in-Time Feature Join

Safe statements:

  • "I will anchor this decision to measurable SLOs and rollback criteria before production promotion."
  • "If time is short, I can zoom into the control plane versus data plane trade-off that drives most ML platform incidents."
Design ML Platform - System Design | WinJob | WinJob