Design API Gateway

What is an API Gateway?

An API gateway is the controlled front door for every HTTP/gRPC call entering your platform. Products like Kong, AWS API Gateway, and Apigee converge authentication, routing, throttling, observability, and protocol bridging so backend teams ship services without re-implementing edge policy on each repo.

Planes you must name

Data plane: stateless (or lightly stateful) proxies that terminate TLS, match routes, execute plugin chains, and forward to upstream clusters with strict timeouts.

Control plane: stores route definitions, consumer credentials, rate-limit policies, and certificate metadata; pushes versioned snapshots to data-plane nodes.

Why interviewers ask this

They want to see you treat the gateway as a policy enforcement point, not a fancy load balancer. Strong answers quantify added latency budget (typically 5–15 ms p99 in-region), explain multi-tenant isolation, and separate config propagation from request hot path.

Scale anchors for this answer

Assume 120K RPS aggregate ingress, 4 KB average request+response, 99.95% gateway availability, and p99 proxy latency < 40 ms excluding upstream. State explicit degradations: shed analytics plugins before auth, never silently bypass OAuth on paid tiers.

Articulate invariants: every request carries X-Request-Id, consumer identity is resolved once per request, and route config version is stamped on responses for debug.

Interview checkpoint

Open by separating data plane (stateless proxies: TLS termination, route match, plugin chain, upstream forward) from control plane (route/credential/policy store pushing versioned snapshots). Then frame the gateway as a policy enforcement point, not a fancy load balancer, and put a latency budget on it — p99 < 40ms excluding upstream — so the interviewer sees you treat every added edge hop as a cost.