Problem Statement: Immersive Video Is a Geometry Problem, Not Just a Streaming Problem
Frames 360-degree video as a spherical media problem where projection, tiling, and viewport-aware delivery dominate the architecture.
Problem statement
Design a platform that lets creators upload 360-degree footage and lets viewers watch it with free camera control: drag-to-look on a phone or browser, and head-tracked immersion on a VR headset. The system must handle specialized spherical video formats, multi-bitrate encoding, quick rebuffering when a user pans, and optionally stream only the viewer's current field of view to cut bandwidth.
A conventional video pipeline treats a frame as a fixed rectangle that every viewer watches identically. 360-degree video breaks that assumption twice. First, the frame is a projection of a sphere: an equirectangular 3840x1920 frame wastes resolution at the poles and concentrates the viewer's attention on roughly one third of the pixels at any instant. Second, every viewer watches a different window into the same frame, decided by head orientation that can change at 90+ Hz. The result is that naive delivery of a 4K spherical frame at broadcast quality costs 15 Mbps or more per viewer, while the pixels actually visible to one person could be served for far less.
Why the problem is distinctive
Standard adaptive bitrate (ABR) adapts quality to bandwidth. Viewport-adaptive streaming adapts quality to direction: high resolution inside the viewer's field of view, a low-resolution blanket everywhere else. This turns the CDN from a byte cache into a spatial cache: the hot working set is not just popular videos, but popular videos times popular view regions. Encoding becomes a tiling problem: split the sphere into independently decodable tiles, build a bitrate ladder per tile, and describe the spatial layout in the manifest using DASH Spatial Relationship Description or an OMAF-style viewport-dependent profile.
Playback is a rendering problem the platform controls: the client must map a spherical texture onto an inward-facing sphere or cube, re-project it every frame from head-pose quaternions, and keep motion-to-photon latency low enough to avoid simulator sickness. Head tracking is therefore a first-class input: 20 Hz pose telemetry on-device, predicted 100-200 ms ahead on the client to hide segment-fetch latency, and batched to the backend at 1 Hz for analytics.
The four architectural planes
- Ingest plane: resumable upload, validation, projection-aware encoding, tiling, packaging.
- Delivery plane: CDN tiering, tile-level caching, manifest fan-out, DRM.
- Playback plane: spherical renderer, tile selection, pose prediction, buffer scheduling.
- Learning plane: viewport heatmaps, QoE telemetry, ladder tuning, recommendation.
A strong answer keeps these planes separate: a failure in analytics must never stall playback, and a render-loop deadline must never depend on a cloud round trip.
Key Highlights
- •The viewer sees ~30% of the sphere's pixels at once, but naive streaming pays for 100%.
- •Viewport-adaptive delivery converts the CDN into a spatial cache keyed by video and view region.
- •Encoding becomes tiling: independently decodable tiles with per-tile bitrate ladders.
- •Head pose is a 90 Hz input; the client predicts 100-200 ms ahead to hide fetch latency.
- •Keep ingest, delivery, playback, and learning planes isolated so failures do not cross planes.
Section Rescue Kit
Buzzwords to use:
Safe statements:
- "I will separate ingest, delivery, playback, and learning planes before choosing any service."
- "The core optimization is spatial: quality follows the viewer's gaze, not just their bandwidth."