Problem Statement: Two Coupled Real-Time Planes With Opposite Delivery Semantics
Frames the product as a market-data plane fused with a social plane, and shows why their delivery semantics are fundamentally different.
Problem statement
Design a real-time stock ticker and chat platform. Traders and watchers see streaming quotes — last trade, bid, ask, change, volume — for thousands of symbols with sub-second freshness. Around every symbol there is a chat channel where users discuss price movement in real time. The platform must highlight big moves and top gainers, keep quotes accurate under spiky load, and keep chat ordered and durable during the exact moments when price volatility drives message volume up.
This is not one system. It is two coupled real-time planes with opposite delivery semantics. The market-data plane is a high-rate, coalescable, last-value stream: when a symbol prints forty updates in one second, a client only needs the most recent state plus a gap-free sequence to trust it. Intermediate quote states are disposable. The chat plane is the inverse: every message is a durable, ordered fact that must eventually reach every member of a channel. You may coalesce quotes. You may never coalesce chat.
The coupling is what makes the design hard. Price moves cause chat bursts. A surprise earnings print, a halt, or a meme squeeze on one symbol simultaneously spikes quote update rates, subscription churn as users add that symbol, chat message volume in that channel, and reconnect churn as overwhelmed clients drop. Load is correlated across every subsystem at the same instant, and it is concentrated on a tiny set of hot symbols. A design sized for averages will pass every quiet test and fail on the one day users care about most.
Why the problem is distinctive
Three properties separate this from a generic chat or a generic dashboard. First, market data carries exchange-derived ordering: consolidated feeds such as the CTA and UTP tapes in the United States publish sequence-numbered quote and trade messages, and the platform must preserve gap-free ordering per symbol or render prices that never existed. Robinhood's engineering team has publicly described building Kafka-based market-data pipelines to normalize and distribute exchange feeds, and the January 28, 2021 GameStop volatility episode — which coincided with a widely reported Robinhood outage — shows what happens when fan-out and capacity meet a retail frenzy. Second, redistribution of exchange data is governed by licensing contracts with professional versus non-professional subscriber classification and display requirements, so entitlements are a first-class architectural input, not a legal footnote. Third, the social layer adds a compliance surface: broker-dealer communications are subject to recordkeeping and supervision regimes such as SEC Rule 17a-4 and FINRA oversight, which shape retention, export, and moderation architecture.
The four architectural planes
- Ingest plane: exchange and provider feeds, normalization, sequencing, gap detection, failover.
- Fan-out plane: subscription routing, coalescing, batching, WebSocket gateways, snapshot service.
- Social plane: channels, messages, presence, moderation, notifications.
- Governance plane: entitlements, licensing, compliance retention, surveillance, audit.
A strong answer keeps these planes separable: the fan-out plane degrades without corrupting the ingest sequence, the social plane throttles without blocking quotes, and governance checks run on subscription and message paths without becoming the hot path itself.
Key Highlights
- •Quotes are coalescable last-value streams; chat messages are durable ordered facts — never apply the same delivery policy to both.
- •Price spikes correlate load across ingest, fan-out, chat, and reconnect paths at the same instant, concentrated on a few hot symbols.
- •Market data requires gap-free per-symbol sequence preservation derived from exchange feeds.
- •Entitlements and compliance regimes are architectural inputs: licensing classes, recordkeeping, and supervision shape retention and access paths.
- •Four planes: ingest, fan-out, social, governance — each degrades independently.
Section Rescue Kit
Buzzwords to use:
Safe statements:
- "I will separate the market-data plane from the social plane because their delivery semantics are opposites."
- "Before choosing infrastructure, let me define what may be dropped, what must be coalesced, and what must never be lost."