Problem Statement: Content Understanding as a First-Class Platform
Frames classification and tagging as a durable content-understanding platform, not a one-off ML API call.
Problem statement
Design a content classification and tagging system for a large streaming and media platform. The platform ingests video, audio, image, and text metadata for movies, episodic shows, user-generated clips, music tracks, podcasts, and livestream recordings. For every asset it must extract structural metadata (title, language, runtime, captions), run an ML classification pipeline (genre, mood, themes, content maturity, setting, entities), apply a governed taxonomy of tags, blend automated output with editorial curation and user feedback, and publish versioned, queryable metadata to search, recommendation, browse, and personalization.
The central insight is that this is a content-understanding platform with a durable data product at its center, not a stateless inference endpoint. A tag attached to an asset today may be re-evaluated next quarter when a better model ships, corrected by an editor, or disputed by users. Therefore the system must treat classification output as versioned, attributable, and reversible evidence, and the taxonomy itself as a governed schema that evolves without silently corrupting billions of existing labels.
Why the problem is distinctive
A recommendation engine consumes tags; it does not create them. This system sits upstream of search, browse, and personalization, so its errors multiply: a wrong maturity rating can surface adult content to children, a wrong genre tag can silently degrade a whole recommendation vertical, and taxonomy sprawl can make the catalog unsearchable. Three properties make it hard. First, the workload is ML-compute-bound, not API-QPS-bound: the bottleneck is GPU minutes for transcription, vision, and audio analysis, not request latency. Second, the output is inherently subjective and probabilistic, so the design must encode confidence, human review, and disagreement rather than pretending every label is ground truth. Third, the taxonomy is a living graph that changes over time, so the system must reconcile historical labels with evolving definitions without a full manual retag.
The four architectural planes
- Ingestion plane: bulk and streaming intake of assets and their sidecar metadata, normalization, and durable staging.
- Understanding plane: asynchronous ML pipelines for transcription, vision, audio, text, multimodal fusion, and taxonomy mapping.
- Curation plane: editorial review queues, user feedback signals, active learning, and confidence-based routing between automation and humans.
- Serving plane: versioned, queryable metadata for search, recommendation, browse, and personalization, plus reclassification backfill.
A strong answer keeps these planes separate. It lets the understanding plane lag or fail without blocking ingestion, lets the curation plane override automation without corrupting the audit trail, and lets the serving plane read a consistent versioned snapshot even while models are being retrained. The taxonomy and its governance rules are the contract that all four planes share.
Key Highlights
- •The bottleneck is GPU minutes for transcription, vision, and audio analysis, not API request latency.
- •Classification output is versioned, attributed, probabilistic evidence, never a mutable unattributed string.
- •The taxonomy is a governed graph that evolves; historical labels must reconcile with new definitions without manual retag.
- •Four planes: ingestion, understanding (ML), curation (human + feedback), serving.
- •Errors multiply downstream: a wrong maturity rating or genre tag degrades search, browse, and recommendation.
Section Rescue Kit
Buzzwords to use:
Safe statements:
- "I will separate ingestion, ML understanding, human curation, and serving so a slow model never blocks intake."
- "Before picking models, let me define the taxonomy contract, because every plane depends on it."