Design a Multi-User VR Chat

Problem statement

Design a multi-user VR chat platform where users join shared 3D spaces as avatars, move and gesture in real time, speak with proximity-based spatial audio, and manipulate shared virtual objects. The system must provision and discover worlds, form instances, transport avatar motion at interactive latency, mix and route voice, persist identity, friendships, and creator content, and moderate behavior at human scale.

This is not a text chat service scaled up. The product is a latency-bounded distributed simulation. A text message can arrive 800 ms late and nobody notices. A head-turn that renders 250 ms late causes social awkwardness; a positional update that stutters for 500 ms breaks presence; voice that arrives out of order or with a 400 ms gap becomes unusable. The design therefore separates three clock domains: the simulation clock (tens of milliseconds), the voice clock (tens to a hundred milliseconds of buffering), and the platform clock (seconds to minutes for social and persistence workflows).

Why the problem is distinctive

A conventional RTC or messaging backend optimizes message delivery. A VR chat backend must continuously replicate a mutable 3D world state to every participant while each participant's headset renders at 60-120 FPS. That imposes four properties that generic chat designs miss:

1. State is continuous, not event-driven. Every avatar emits pose and animation samples many times per second. There is no natural request/response boundary.
2. The fan-out is quadratic inside a room. N avatars produce N(N-1) observation pairs. A 32-person room is not twice a 16-person room; it is four times the replication work.
3. Audio dominates bandwidth and perception. Voice is the social carrier. Pose can be smoothed and predicted; voice gaps are immediately noticeable.
4. The client is untrusted but latency-critical. Anti-cheat and custody of world objects demand server authority, but waiting a round trip for every motion is impossible, so prediction and reconciliation are mandatory.

Public operating baseline

The category is operationally real. VRChat has publicly reported millions of monthly active users and instance-based rooms with per-instance user caps, plus a scripting layer (Udon) and avatar IK systems. Linden Lab's Second Life has run a region-grid architecture for two decades, where each region is simulated by a dedicated simulator process. Rec Room operates cross-platform rooms built on Unreal Engine networking. Improbable publicly demonstrated SpatialOS, which partitions a single large world across many worker processes with transferable entity authority. These are public descriptions used as design context; every scale number in this answer that is not attributed is an explicit design assumption.

The four architectural planes

1. Simulation plane: room servers that own authoritative world state, tick the simulation, reconcile client inputs, and replicate snapshots.
2. Voice plane: capture, encoding, selective forwarding, spatial rendering, and the signaling that binds voice rooms to instances.
3. Platform plane: identity, worlds catalog, matchmaking, friends, presence projections, economy, and creator tooling.
4. Trust plane: moderation, reporting, evidence handling, age and jurisdiction policy, and safety features such as personal space and blocking.

A strong interview answer keeps these planes separate. The platform plane may degrade for minutes while users keep talking; the voice plane may shed fidelity while simulation stays authoritative; but the simulation plane must never stall, because a stalled simulation is a frozen world.