Design Video Content Moderation - System Design | WinJob

Video Moderation Context and Trust-Safety Goals

Frame the problem as trust & safety at UGC scale, not "build an ML model." YouTube/TikTok/Meta-class platforms ingest 4M video uploads/day, average 3 minutes, 500M DAU, and must block violence, CSAM, hate, spam, and IP violations before harmful reach.

Platform constraints

Quarantine-by-default: catalog and CDN treat new assets as UNLISTED_QUARANTINED until visibility_decision event commits.
Multimodal: visual frames, ASR transcript, title/description hashtags, audio fingerprint, creator trust tier.
Explainability: regulators and appeals need ruleId + modelVersion, not a black-box score.

Metrics interviewers expect

Metric	Target
time-to-quarantine p95	< 60s VOD
auto-decision rate	70–85%
false-positive rate	< 0.5% on allow
human review SLA (high risk)	< 4h

Why this is hard

Viral velocity: a 60s video can reach 1M views in 20 minutes if public. Moderation latency is a product SLO, not offline batch analytics.

Implementation sketch

javaOne Dark Pro

1 public enum Visibility { QUARANTINED, PUBLIC, LIMITED, REMOVED }
2 public record AssetDecision(String assetId, Visibility v, String ruleId) {}

pythonOne Dark Pro

1 @dataclass(frozen=True)
2 class ModCtx1:
3     asset_id: str
4     topic: str = "context"

typescriptOne Dark Pro

1 export interface ModCtx1 {
2   assetId: string;
3   topic: "context";
4 }

Operational notes (context)

Dashboard: queue lag, quarantine rate, purge failures.
Runbook: fail-closed when inference error rate > 1% for 5m.
On-call pairs with trust-safety legal for CSAM spikes.

Scale anchors

4M uploads/day, 3 min average, 500M DAU.
5% human routing → ~200k reviewer decisions/day without automation improvements.
Frame peak ~25k/s; GPU fleet ~800 at peak with batching.

Interview positioning

Tie Video Moderation Context and Trust-Safety Goals back to time-to-quarantine and explainable policy—not model F1 in a vacuum.

Why interviewers care

Video Content Moderation interviews reward crisp scope, explicit trade-offs, and failure stories—not generic microservice diagrams.

Interview checkpoint

Name one failure story for Video Moderation Context and Trust-Safety Goals that proves you understand real outages, not happy-path diagrams.

Key Highlights

•Quarantine-by-default until a durable moderation decision reaches catalog and CDN
•Harm metrics: time-to-quarantine p95, reviewer SLA, false-positive rate
•Multimodal signals: frames, ASR transcript, metadata, creator trust

1	public enum Visibility { QUARANTINED, PUBLIC, LIMITED, REMOVED }
2	public record AssetDecision(String assetId, Visibility v, String ruleId) {}

1	@dataclass(frozen=True)
2	class ModCtx1:
3	asset_id: str
4	topic: str = "context"

1	export interface ModCtx1 {
2	assetId: string;
3	topic: "context";
4	}