Design Debug Container System - System Design | WinJob

Problem Statement: Governed Debug Container Platform

Engineers debugging Kubernetes need governed attach without shipping shells in prod images.

Problem framing

Engineers need shells, packet capture, and debugger attach on distroless workloads without baking tools into prod images
Platform must orchestrate EphemeralContainers (Kubernetes 1.23+) or copy-target debug pods with strict TTL and audit
Telepresence-class traffic interception and Skaffold file sync are adjacent dev-loop concerns the control plane should integrate with, not reinvent
Target: p95 session start < 15s; admission webhook < 80ms p99; 100% sessions logged to SIEM

Design choices

Session API issues signed tokens; agents on cluster nodes or API server subresource create ephemeral containers
Profile catalog pins allowed images, capability drops, and namespace modes (pid, net, ipc, uts)
Policy engine enforces environment, owner, ticket ID, and concurrent session caps before kube admits
Audit pipeline streams kubectl-equivalent records to immutable store for SOC review

Deep dive

ephemeral debug containers, kubectl debug, Telepresence-style traffic swap, Skaffold dev loops, break-glass RBAC, and distroless attach paths. Google, Shopify, and large Kubernetes adopters standardized on ephemeral debug containers after distroless became default—kubectl debug -it pod --image=netshoot --target=app shares the app container namespaces without mutating the prod image. Your platform wraps that with governance: only approved profiles, automatic teardown, and integration with IDE attach (dlv, jdwp) via port-forward brokers.

javaOne Dark Pro

1 public final class SessionGate {
2   public boolean allow(String env, boolean breakGlass, int active) {
3     if ("prod".equals(env) && !breakGlass) return false;
4     return active < 3;
5   }
6 }

pythonOne Dark Pro

1 from dataclasses import dataclass
2  
3 @dataclass
4 class DebugSession:
5     target_pod: str
6     profile: str
7     share_pid: bool
8     share_net: bool
9  
10 def ttl_expired(age_s: int, max_ttl: int) -> bool:
11     return age_s > max_ttl

typescriptOne Dark Pro

1 export interface DebugProfile {
2   image: string;
3   capabilities: string[];
4   maxTtlMinutes: number;
5 }
6  
7 export function canAttach(env: string, approved: boolean): boolean {
8   if (env === "production" && !approved) return false;
9   return true;
10 }

Interviewer positioning

Lead with never mutate prod images for debugging and break-glass is audited, time-boxed, and capability-bounded. Mention how Kubernetes EphemeralContainer API differs from legacy kubectl exec into a shell that was never supposed to exist.

How to open this one

The framing that lands for a governed debug-container platform is giving engineers ephemeral, audited access to attach a debug toolset to a running workload — without baking tools into production images. Lead with ephemeral containers (kubectl debug) scoped by RBAC, time-bound, and fully audited, and the failure story that proves it: an engineer needs to debug a distroless prod pod that has no shell, and an ephemeral debug container attaches the tools safely instead of someone shipping a debug build to prod. That shows you understand the goal is least-privilege, audited break-glass debugging, not putting debug tools in production.

Key Highlights

•EphemeralContainer keeps prod images distroless
•Profiles bound capabilities and TTL
•Audit every attach with ticket correlation

1	public final class SessionGate {
2	public boolean allow(String env, boolean breakGlass, int active) {
3	if ("prod".equals(env) && !breakGlass) return false;
4	return active < 3;
5	}
6	}

1	from dataclasses import dataclass
2
3	@dataclass
4	class DebugSession:
5	target_pod: str
6	profile: str
7	share_pid: bool
8	share_net: bool
9
10	def ttl_expired(age_s: int, max_ttl: int) -> bool:
11	return age_s > max_ttl

1	export interface DebugProfile {
2	image: string;
3	capabilities: string[];
4	maxTtlMinutes: number;
5	}
6
7	export function canAttach(env: string, approved: boolean): boolean {
8	if (env === "production" && !approved) return false;
9	return true;
10	}