Design Traffic Management - System Design | WinJob

Problem Statement: Platform Traffic Management

Design a central traffic management platform that sits in front of hundreds of microservices and enforces L7 routing, load balancing, rate limiting, circuit breaking, and graceful degradation without every team reimplementing Envoy/NGINX rules.

Companies like Envoy, NGINX, and HAProxy ask this to test whether you understand data plane vs control plane, connection pooling, health-aware routing, and overload protection under multi-region scale.

Core user journeys

Platform SRE: publish routing policies, drain unhealthy pools, flip traffic during incidents.
Service owner: register upstream clusters, set per-route timeouts/retries, request canary weights.
Security: enforce global rate limits, WAF integration, mTLS at the edge.

Success criteria

Policy propagation < 5s p99 to all edge proxies.
Failover to healthy AZ < 30s after pool health drops.
Rate-limit decisions < 2ms p99 at 100k RPS edge.

javaOne Dark Pro

1 public final class CircuitBreaker {
2   private int consecutiveFailures;
3   public boolean allowRequest(int threshold) {
4     return consecutiveFailures < threshold;
5   }
6   public void recordFailure() { consecutiveFailures++; }
7   public void recordSuccess() { consecutiveFailures = 0; }
8 }

pythonOne Dark Pro

1 def token_bucket_allow(tokens: float, rate: float, burst: float, cost: float = 1.0) -> tuple[bool, float]:
2     tokens = min(burst, tokens + rate)
3     if tokens >= cost:
4         return True, tokens - cost
5     return False, tokens

typescriptOne Dark Pro

1 export function pickHost<T extends { weight: number; healthy: boolean }>(hosts: T[]): T | null {
2   const healthy = hosts.filter((h) => h.healthy);
3   if (healthy.length === 0) return null;
4   const total = healthy.reduce((s, h) => s + h.weight, 0);
5   let r = Math.random() * total;
6   for (const h of healthy) {
7     r -= h.weight;
8     if (r <= 0) return h;
9   }
10   return healthy[healthy.length - 1];
11 }

Interview Focus

Quantify edge RPS, connection pools, and proxy CPU before picking Envoy vs NGINX
Explain health-aware routing and drain semantics
Show token-bucket vs leaky-bucket rate limits with failure behavior
Describe xDS fan-out, fail-static, and circuit breaker thresholds

How to open this one

The framing that signals depth on traffic management is programmable routing as the lever for safe releases and resilience: weighted splits for canaries, header-based routing, retries, timeouts, and circuit breakers at the edge and in the mesh. Lead with how a canary weight ramp is gated by SLOs, and the failure story that proves it: a retry storm amplifies a partial outage because retries had no budget or circuit breaker, turning a blip into a cascade. That shows you understand traffic management is about controlling blast radius, not just spreading load.

Key Highlights

•Separate traffic control plane from Envoy/NGINX data plane
•Health-aware routing with explicit drain semantics
•Layer local burst limits under global quota enforcement
•Fail static on xDS loss; never black-hole production

1	public final class CircuitBreaker {
2	private int consecutiveFailures;
3	public boolean allowRequest(int threshold) {
4	return consecutiveFailures < threshold;
5	}
6	public void recordFailure() { consecutiveFailures++; }
7	public void recordSuccess() { consecutiveFailures = 0; }
8	}

1	def token_bucket_allow(tokens: float, rate: float, burst: float, cost: float = 1.0) -> tuple[bool, float]:
2	tokens = min(burst, tokens + rate)
3	if tokens >= cost:
4	return True, tokens - cost
5	return False, tokens