Design a Livestream Chat Service

Hard45 min
1 / 30
understanding•10 min read

Problem Statement: Chat as a Fan-Out Problem, Not a Messaging App

Frames livestream chat as a bursty, read-amplified pub/sub system with a trust-and-safety plane, distinct from team or peer messaging.

Problem statement

Design the chat overlay for a livestream platform: thousands to millions of concurrent viewers per channel, a text stream that bursts during hype moments, optional moderation and slow mode, and emotes plus reactions. The product surface looks simple; the system underneath is one of the hardest fan-out problems in social infrastructure because the reader-to-writer ratio is inverted compared to almost any other chat product.

In team chat (Slack) or peer messaging (WhatsApp), the fan-out of one message is small: a handful of members. In livestream chat, one message from the streamer must reach millions of connected clients within a couple of seconds, while tens of thousands of viewer messages per second compete for ingestion, moderation, and delivery on the same channel. The defining ratio is therefore fan-out amplification: deliveries per second equals the sum over every live channel of its message rate multiplied by its connected viewers. That single product of two variables dominates the architecture, the cost model, and the failure modes.

The second defining property is burstiness. Chat rate is not平稳 traffic; it spikes with raids, drops, goals, clutch plays, and giveaways. A channel that idles at 200 messages per second can hit 20,000 within five seconds. The system must absorb a 100x ingest spike and a correlated fan-out spike without dropping the streamer's own messages, without letting spam win, and without a reconnect storm when a gateway node dies mid-hype.

The third defining property is abuse. Livestream chat is pseudonymous, public, and emotionally charged, which makes it a prime target for coordinated spam and harassment. Moderation is not a feature bolted on at the end; it is an inline architectural plane with latency budgets, because every viewer message passes through it before fan-out.

Why this is not a group chat redesign

A naive design treats the channel as a room where every member receives every message. That collapses at livestream scale for three reasons: write amplification into delivery sockets, egress cost, and client render capacity. A phone cannot meaningfully render 20,000 messages per second; therefore large channels must coalesce, batch, and sample, while small channels keep full-fidelity push. The architecture must tier channels by live audience and apply different delivery contracts per tier, rather than one global broadcast semantic.

The four planes

  1. Ingest plane: connection gateways, authentication, per-user and per-channel rate limits, idempotent acceptance, and schema validation.
  2. Safety plane: synchronous cheap filters, asynchronous classifiers, human review queues, enforcement ladders, and appeals.
  3. Fan-out plane: room routing, tiered delivery, coalescing windows, priority lanes, and edge distribution.
  4. Experience plane: emotes and reactions, presence, chat history window, slow mode UX, and client render sampling.

A strong interview answer keeps these planes separate so that a moderation outage degrades the safety plane into a documented safe mode without silently turning the fan-out plane into an unmoderated broadcast, and so that a fan-out overload sheds sampled traffic before it ever touches the streamer's lane.

Key Highlights

  • •Fan-out amplification (channel rate x connected viewers) is the dominant workload, not message ingestion.
  • •Livestream chat inverts the usual ratio: millions of readers per writer, unlike team or peer chat.
  • •Bursts of 100x within seconds are normal (raids, drops, goals), so steady-state sizing is meaningless alone.
  • •Moderation is an inline architectural plane with its own latency budgets, not a post-hoc feature.
  • •Large channels must coalesce and sample; small channels keep full-fidelity push - one semantic does not fit all.
Lead With the Fan-Out Product
Open by writing deliveries/sec = sum(channel_rate x viewers). It instantly shows you sized the broadcast problem, not a group chat, and it sets up tiering, coalescing, and egress cost for the rest of the interview.
Do Not Broadcast Per-Message to Millions
Sending each message individually to every socket of a 3M-viewer channel at 20K msg/s is 60B socket writes per second. That design dies on egress and client render capacity; coalescing and sampling are mandatory at that tier.

Section Rescue Kit

Buzzwords to use:

Fan-Out AmplificationCoalesced Delivery

Safe statements:

  • "Let me separate ingest, safety, fan-out, and experience before choosing any technology."
  • "The channel audience distribution is bimodal, so I will define delivery tiers before discussing databases."
Design a Livestream Chat Service - System Design | WinJob | WinJob