Design a Live “Hints & Tips” Chat Overlay

Problem statement

Design a live “Hints & Tips” overlay for an activity app — a coding practice platform or a puzzle game — where verified instructors and community contributors push short, contextual suggestions that appear on top of the user’s active screen without interrupting the activity. Instructors run live sessions where thousands of learners watch the same problem and receive guidance in real time. Community members contribute tips asynchronously, and the best tips are upvoted, highlighted, and served later to anyone who reaches the same context.

The trap in this question is treating it as “design a chat system.” Chat is a poor model here. A chat room is a destination; this overlay is an augmentation layer. Nobody opens the overlay to socialize. The learner is mid-puzzle, mid-compile, mid-lesson, and the system must decide four things a chat system never asks: Which tip is relevant to the exact step this user is on right now? Who is allowed to push into this user’s attention? How do we deliver it in under a second without degrading the host app’s frame budget? And how do we stop spam without adding a second of latency to a verified instructor?

Why the problem is distinctive

The unit of delivery is not a conversation — it is a context binding. A tip addresses a selector such as course=algorithms-101, problem=two-sum, step=edge-cases, and optionally a state predicate like “user has failed two test attempts.” The client reports its current context; the platform resolves which published tips match; and only those are streamed to the overlay. When the learner advances to the next step, the subscription set changes. This makes the read path a matching problem and the write path a fanout problem, joined by a moderation gate.

Public evidence shows the category is real at scale. Discord’s engineering team has described storing trillions of messages and migrating that store from Cassandra to ScyllaDB, serving hundreds of millions of registered users. Twitch operates live channels whose chat sustains extreme message-rate spikes during global events, and its moderation tooling runs at the same scale. Duolingo reported 116.3 million monthly active users in its fourth-quarter 2024 earnings release, and its product is built on contextual hints, streaks, and nudges layered over a lesson flow. Stack Overflow’s vote-ranked answers are the asynchronous ancestor of the “best tip floats to the top” requirement. These are cited company figures used as context; every scale number used for capacity planning in this answer is an explicit design assumption unless tied to such a source.

The four architectural planes

1. Activity plane: the host app (IDE, puzzle engine, lesson player) emits context events — screen, step, attempt count, errors — and renders the overlay without blocking its own main loop.
2. Overlay plane: the client-side overlay runtime owns lifecycle (show, pin, collapse, dismiss), snapshot-plus-delta synchronization, and an attention budget so the host activity is never starved.
3. Authoring plane: verified instructors, community contributors, and ML-suggested hint generators publish tips with context selectors, TTLs, and priority classes.
4. Trust plane: moderation gates, author reputation, rate limits, enforcement ladder, and audit — architected so verified-author latency and crowd-safety are tuned independently.

A strong interview answer keeps these planes separate. A moderation outage must not freeze an instructor’s live session. A fanout lag must not corrupt the activity app. A context event must never be able to inject a tip.

Design assumptions for this answer

Unless a number is tied to a public source above, it is a stated design assumption: 5M MAU, 800K DAU, 150K peak concurrent users, 500 concurrent live instructor sessions at peak, tip authoring at 25 tips/second at peak, and 50K overlay deliveries/second provisioned. The rest of this answer sizes every component from these numbers.