Design WebSocket Gateway - System Design | WinJob

What is a WebSocket Gateway?

A WebSocket gateway is the connection edge of a real-time platform: it terminates millions of long-lived TCP sessions, authenticates clients, enforces channel ACLs, and routes publish events to the right subscribers. Products like Pusher, Ably, and Slack expose this layer so application backends stay HTTP-centric while clients receive sub-second updates.

Core responsibilities

Session plane: accept upgrades, keep heartbeats, detect half-open links, and assign each socket a stable connection_id plus routing metadata (region, shard, protocol version).

Subscription plane: map logical channels (room:42, user:9, presence:lobby) to connection sets with refcounted membership and permission checks on every subscribe/unsubscribe.

Publish plane: ingest events from REST or internal buses, fan out to local sockets first, then to peer gateway nodes via a cross-node pub/sub bus (Redis, NATS, or dedicated mesh).

Why interviewers ask this

The question isolates stateful connection scaling from generic API design. Strong candidates explain sticky routing, backpressure during reconnect storms, and why you never treat a gateway as a stateless microservice behind round-robin without session affinity.

Scale anchors

Model 10M concurrent connections per region, 500K events/s publish throughput, and p99 delivery < 100ms for online subscribers. Name explicit degradations: coalesce presence, drop typing, never silently drop subscribed channel events without client-visible error codes.

State the invariants: at-most-once vs at-least-once per channel class, ordering scope (per-channel FIFO), and auth token binding to connection lifecycle.

Connect Problem Statement & Context to measurable gateway SLOs and name the first metric you would alert on.

javaOne Dark Pro

1 public boolean tryAcquire(String tenantId) {
2   return rateLimiter.tryConsume(tenantId, 1);
3 }

pythonOne Dark Pro

1 def try_acquire(tenant_id: str) -> bool:
2     return limiter.consume(tenant_id, 1)

typescriptOne Dark Pro

1 export function tryAcquire(tenantId: string): boolean {
2   return rateLimiter.consume(tenantId, 1);
3 }

Why interviewers care

WebSocket Gateway interviews reward crisp scope, explicit trade-offs, and failure stories—not generic microservice diagrams.

Interview checkpoint

Name one failure story for Problem Statement & Context that proves you understand real outages, not happy-path diagrams.

Key Highlights

•Gateway owns long-lived state; app servers publish asynchronously
•Channels are logical; connections are physical and shard-local
•Cross-node fanout is the scaling bottleneck after single-node limits
•Reconnect storms dominate incident history—design admission control first

1	public boolean tryAcquire(String tenantId) {
2	return rateLimiter.tryConsume(tenantId, 1);
3	}

1	def try_acquire(tenant_id: str) -> bool:
2	return limiter.consume(tenant_id, 1)

1	export function tryAcquire(tenantId: string): boolean {
2	return rateLimiter.consume(tenantId, 1);
3	}