Design AutoML Platform - System Design | WinJob

Problem Framing: AutoML for Business Users

AutoML study lifecycle from CSV upload to deployable endpoint with model card.

Interviewers expect you to treat an AutoML platform as a distributed scheduling and governance problem first and a collection of algorithms second. The hard parts are not picking XGBoost versus a neural net—they are preventing label leakage, enforcing search budgets, and proving the exported pipeline matches what scored on the leaderboard.

Design lens 1

AutoML study lifecycle from CSV upload to deployable endpoint with model card. In practice this means explicit metrics, rollback triggers, and an on-call runbook entry—not hand-wavy “we'll monitor it.”

Design lens 2

Why Google, DataRobot, and H2O ask this for platform + search + governance depth. In practice this means explicit metrics, rollback triggers, and an on-call runbook entry—not hand-wavy “we'll monitor it.”

Design lens 3

Separating citizen data scientist sandboxes from regulated production studies. In practice this means explicit metrics, rollback triggers, and an on-call runbook entry—not hand-wavy “we'll monitor it.”

Operational signals to cite aloud

Study queue P99 under 90s for 50-trial tabular jobs; hours only for approved deep-learning searches.
Trial success rate target 94% excluding user data errors; platform faults < 2% of trials.
Leaderboard freshness within 30s of trial completion for UI progress bars.
Search efficiency = useful trials / GPU-hour; alert if median drops below 0.4 for tree searches.

Failure modes you should volunteer

Leaked validation into training folds, runaway Bayesian search burning GPU budget, champion model promoted without fairness constraints, and PII columns auto-engineered into features without policy tags.

Code: study admission (sketch)

javaOne Dark Pro

1 // Conceptual Java — study admission with quota + idempotency
2 record StudyRequest(String idempotencyKey, String workspaceId, long budgetGpuSeconds) {}
3 class StudyAdmissionService {
4   Study admit(StudyRequest req) {
5     if (quotaService.remaining(req.workspaceId()) < req.budgetGpuSeconds())
6       throw new QuotaExceededException();
7     return studyRepo.upsertByIdempotencyKey(req.idempotencyKey(), req);
8   }
9 }

pythonOne Dark Pro

1 # Conceptual Python — budget fuse on active trials
2 def projected_spend(study_id: str) -> int:
3     trials = trial_repo.list_running(study_id)
4     return sum(t.gpu_seconds_elapsed + t.eta_gpu_seconds for t in trials)
5  
6 def should_fuse(study_id: str, cap: int) -> bool:
7     return projected_spend(study_id) > int(cap * 1.1)

typescriptOne Dark Pro

1 // Conceptual TypeScript — idempotent create study handler
2 export async function createStudy(req: Request) {
3   const key = req.headers.get("Idempotency-Key");
4   if (!key) throw new BadRequestError("missing idempotency key");
5   const existing = await db.study.findByKey(key);
6   if (existing) return Response.json(existing, { status: 200 });
7   const body = await req.json();
8   const study = await admission.admit({ ...body, idempotencyKey: key });
9   return Response.json(study, { status: 201 });
10 }

Why interviewers care

AutoML Platform interviews reward crisp scope, explicit trade-offs, and failure stories—not generic microservice diagrams.

Interview checkpoint

Name one failure story for Problem Framing: AutoML for Business Users that proves you understand real outages, not happy-path diagrams.

Key Highlights

•Problem Framing: AutoML for Business Users: AutoML study lifecycle from CSV upload to deployable endpoint with model card. (emphasis 1).
•Problem Framing: AutoML for Business Users: Why Google, DataRobot, and H2O ask this for platform + search + governance depth. (emphasis 2).
•Problem Framing: AutoML for Business Users: Separating citizen data scientist sandboxes from regulated production studies. (emphasis 3).

1	// Conceptual Java — study admission with quota + idempotency
2	record StudyRequest(String idempotencyKey, String workspaceId, long budgetGpuSeconds) {}
3	class StudyAdmissionService {
4	Study admit(StudyRequest req) {
5	if (quotaService.remaining(req.workspaceId()) < req.budgetGpuSeconds())
6	throw new QuotaExceededException();
7	return studyRepo.upsertByIdempotencyKey(req.idempotencyKey(), req);
8	}
9	}

1	# Conceptual Python — budget fuse on active trials
2	def projected_spend(study_id: str) -> int:
3	trials = trial_repo.list_running(study_id)
4	return sum(t.gpu_seconds_elapsed + t.eta_gpu_seconds for t in trials)
5
6	def should_fuse(study_id: str, cap: int) -> bool:
7	return projected_spend(study_id) > int(cap * 1.1)

1	// Conceptual TypeScript — idempotent create study handler
2	export async function createStudy(req: Request) {
3	const key = req.headers.get("Idempotency-Key");
4	if (!key) throw new BadRequestError("missing idempotency key");
5	const existing = await db.study.findByKey(key);
6	if (existing) return Response.json(existing, { status: 200 });
7	const body = await req.json();
8	const study = await admission.admit({ ...body, idempotencyKey: key });
9	return Response.json(study, { status: 201 });
10	}