Design Model Compression - System Design | WinJob

Problem Statement: Enterprise Model Compression

Design a production model compression platform that shrinks neural networks via quantization, pruning, and distillation while enforcing accuracy gates before edge or mobile deployment. This section uses a accuracy-gated lens on turning FP32 teacher checkpoints into deployable INT8/FP16 bundles with measurable accuracy and latency contracts.

Why Google, Apple, and Qualcomm interviews probe here

Mobile and edge teams must ship smaller, faster models without blowing accuracy budgets. Interviewers expect you to separate calibration data governance, batch compression workers, and hardware-specific compilation from the training platform. Hand-waving “quantize to INT8” fails when asked about slice regressions, signed artifacts, or spot GPU preemption.

Operational detail you should voice aloud

State numeric assumptions: 12K compression jobs/day, 800 new baselines/month, 45 peak submit QPS, P99 accept 180 ms, 28 TB/day artifact egress, and 4–8× weight reduction targets validated on golden eval—not parameter math alone.

Failure modes worth volunteering

Stale calibration sets, compile failures on unsupported ops, accuracy cliffs on rare language slices, spot instance preemption without checkpoint resume, and registry promotion of unsigned bundles. For each, name detection (calibration hash mismatch, compile DLQ, slice alert, job heartbeat loss, signature verify failure) and mitigation (snapshot pinning, alternate opset fallback, QAT branch, resume from manifest checkpoint, cosign gate).

Whiteboard checkpoint

Draw submit API → workflow engine → calibration cache → GPU workers → benchmark harness → signing → registry on one line. Label where accuracy gates run and where immutable artifacts land in object storage.

Implementation snippets (model compression)

javaOne Dark Pro

1 public record CompressionJobId(String workspaceId, String jobId) {}
2 public enum CompressionState {
3   QUEUED, CALIBRATING, COMPRESSING, BENCHMARKING, PROMOTED, FAILED
4 }

pythonOne Dark Pro

1 @dataclass(frozen=True)
2 class CompressionJobId:
3     workspace_id: str
4     job_id: str
5  
6 class CompressionState(str, Enum):
7     QUEUED = "queued"
8     CALIBRATING = "calibrating"
9     COMPRESSING = "compressing"
10     BENCHMARKING = "benchmarking"
11     PROMOTED = "promoted"
12     FAILED = "failed"

typescriptOne Dark Pro

1 export interface CompressionJobId {
2   workspaceId: string;
3   jobId: string;
4 }
5 export type CompressionState =
6   | "queued"
7   | "calibrating"
8   | "compressing"
9   | "benchmarking"
10   | "promoted"
11   | "failed";

Section-specific depth (sec-01)

For Problem Statement: Enterprise Model Compression, tie compression as a batch dag—not an ad-hoc notebook export to measurable accuracy/footprint SLIs, explain how technique router picks ptq, qat, pruning, or distillation per hardware profile changes GPU-hour estimates, and show how accuracy regression gates block promotion to model registry affects registry promotion contracts—not generic “use quantization” slogans.

Why interviewers care

Model Compression interviews reward crisp scope, explicit trade-offs, and failure stories—not generic microservice diagrams.

Interview checkpoint

Name one failure story for Problem Statement: Enterprise Model Compression that proves you understand real outages, not happy-path diagrams.

Key Highlights

•Compression as a batch DAG—not an ad-hoc notebook export
•Technique router picks PTQ, QAT, pruning, or distillation per hardware profile
•Accuracy regression gates block promotion to model registry
•Output artifacts: ONNX/TFLite/CoreML packages plus benchmark report

1	public record CompressionJobId(String workspaceId, String jobId) {}
2	public enum CompressionState {
3	QUEUED, CALIBRATING, COMPRESSING, BENCHMARKING, PROMOTED, FAILED
4	}

1	@dataclass(frozen=True)
2	class CompressionJobId:
3	workspace_id: str
4	job_id: str
5
6	class CompressionState(str, Enum):
7	QUEUED = "queued"
8	CALIBRATING = "calibrating"
9	COMPRESSING = "compressing"
10	BENCHMARKING = "benchmarking"
11	PROMOTED = "promoted"
12	FAILED = "failed"

1	export interface CompressionJobId {
2	workspaceId: string;
3	jobId: string;
4	}
5	export type CompressionState =
6	\| "queued"
7	\| "calibrating"
8	\| "compressing"
9	\| "benchmarking"
10	\| "promoted"
11	\| "failed";