Design Distributed Tracing

Problem Statement: End-to-End Trace Observability

Design an enterprise-grade distributed tracing platform (OpenTelemetry-native, Jaeger/Zipkin-compatible) that SRE and platform teams use to debug cross-service latency (think Uber Jaeger origins, Zipkin at Twitter scale, Datadog APM trace intake).

The core problem a tracing platform solves is correlation across process boundaries: a single user request fans out into dozens of service calls, and without a shared trace context you have a pile of disconnected logs and no way to answer "which hop made checkout slow?" A trace stitches those hops into one causal tree of spans — each span a timed operation with a trace_id, span_id, parent_span_id, and attributes — propagated end to end via the W3C traceparent header. The platform's job is to ingest those spans at scale, assemble them into traces, and let an engineer search and visualize them in seconds during an incident.

The defining constraint is volume versus value: at ~11.5M spans/sec you cannot store everything affordably, yet the traces you most need are the rare slow or errored ones. That tension drives every later decision — sampling strategy (head vs tail), columnar storage, retention tiers, and high-cardinality index control. Anchor the design on Uber's Jaeger, Twitter's Zipkin, and OpenTelemetry as the de-facto wire standard, and frame success as: an SRE goes from "p99 spiked" to the offending span in under a minute, while ingest never becomes the outage it is meant to diagnose.

Interview checkpoint

Frame it as a correlation-at-scale problem, not "collect logs": one request becomes a causal span tree stitched by a propagated trace_id. Then name the central tension — 11.5M spans/sec is too much to keep, but the valuable traces are the rare slow/errored ones — because sampling, storage, and indexing all flow from it.