Design a Trending Topics Service

Problem Statement

Design a Trending Topics Service that identifies trending keywords, hashtags, and entities in real time across user-generated content, ranks them by velocity and engagement, and serves region- and topic-based trend lists to hundreds of millions of users.

This is not a batch analytics problem. A trending topic is defined by acceleration, not absolute volume. The word "earthquake" may appear in 10,000 posts per hour during a seismic event but only 200 per hour on a normal day. The system must detect that 50× velocity spike within minutes, not hours. Meanwhile, a celebrity hashtag that accumulates 500,000 mentions steadily over 12 hours is popular but not necessarily trending in the current window.

Why This Problem Is Hard

A social media platform with 200 million daily active users generating 500 million posts per day produces roughly 5,800 posts per second on average and 30,000+ posts per second at peak. Each post may contain multiple hashtags, mentions, keywords, and entities. The system must:

1. Ingest every post in real time without back-pressure stalling the main feed pipeline.
2. Extract entities: hashtags (#), @mentions, named entities (people, places, events), and keyword n-grams.
3. Aggregate frequencies over sliding time windows (1-minute, 5-minute, 1-hour, 24-hour) per region.
4. Score each candidate using velocity, acceleration, engagement weight, and freshness decay.
5. Rank and serve the top-K trends per region, refreshed every 30–60 seconds.
6. Filter spam, bot amplification, and policy-violating content before surfacing trends.

The fundamental tension is between accuracy and throughput. Exact counting at 30K events/sec across 10 regions and millions of distinct tokens requires enormous memory. Approximate data structures—Count-Min Sketch, Space-Saving, Heavy Hitters—trade a bounded error rate for 100–1000× memory reduction. A strong interview answer makes this trade-off explicit and quantifies the error bounds.

The Four Architectural Planes

1. Ingestion Plane: Receives raw post events from the content platform via a durable message bus (Kafka). Handles schema validation, deduplication, and partitioning.
2. Processing Plane: Stream processors (Flink, Kafka Streams, or equivalent) run entity extraction, windowed aggregation, scoring, and anomaly detection.
3. Serving Plane: Low-latency read path. Caches top-K trend lists per region in Redis/Memcached. Serves API requests in under 50ms p99.
4. Governance Plane: Content moderation filters, spam detection, bot amplification guards, and compliance controls that gate what appears in trends.

A strong answer keeps these planes separate. The processing plane can lag by 10 seconds without the serving plane returning stale-but-cached results. The governance plane can block a trend without stopping the pipeline. The ingestion plane can shed load without corrupting counts.

Public Operating Baselines Versus Design Assumptions

Twitter (now X) publicly reported processing over 500 million tweets per day at peak, with trending topics computed across global and regional partitions. Reddit's ranking algorithms are open-source and use time-decay scoring. YouTube's trending tab processes billions of daily views with view-velocity signals. These are cited public figures that establish the category is operationally real.

For capacity planning, this answer explicitly assumes a mature platform with 200 million DAU, 500 million posts per day, 30,000 posts/sec at peak, 10 geographic regions, and a 5× event multiplier. Unless a number is tied to a citation, it is a stated design assumption.