Design Prompt Management - System Design | WinJob

Problem Statement: Enterprise Prompt Management

Design a central prompt management platform that versions templates, validates variables, evaluates changes, and serves production resolves at inference scale. This section uses a mechanism-first lens on workspace → project → prompt template → immutable version commits.

Why LangChain, Anthropic, and OpenAI interviews probe here

Platform teams need engineers who treat prompts as release artifacts, not copy-paste strings. Hand-waving "we store prompts in Git" fails when asked about 420K resolve QPS, promotion gates, partial composition cycles, or PII leakage in committed bodies.

Operational detail you should voice aloud

State numeric assumptions: 1.8M version commits/day, 48M evaluation invocations/day, 420K peak resolve QPS, P99 resolve 35ms, P99 commit 180ms, and 4.2KB average prompt bodies. Tie each figure to a formula on the whiteboard.

Failure modes worth volunteering

Duplicate commits from retried CI pipelines, resolver serving stale prod during tag cutover, eval workers stuck on judge API timeouts, and search indexing lag hiding the latest approved version. For each, name detection (idempotency collision metric, cache age histogram, eval queue lag alert) and mitigation (Idempotency-Key table, prompt.promoted cache bust, DLQ + sweepers).

Whiteboard checkpoint

Draw commit API + blob store on the left, metadata Postgres + OpenSearch in the center, edge resolver cache on the right. Label where sha256 dedupe saves storage and where tag pointers enable instant rollback.

Implementation snippets (prompt management)

javaOne Dark Pro

1 public record VersionKey(String workspaceId, String versionId) {}
2 public enum PromptTag { DRAFT, STAGING, PRODUCTION }

pythonOne Dark Pro

1 @dataclass(frozen=True)
2 class VersionKey:
3     workspace_id: str
4     version_id: str
5  
6 class PromptTag(str, Enum):
7     DRAFT = "draft"
8     STAGING = "staging"
9     PRODUCTION = "production"

typescriptOne Dark Pro

1 export interface VersionKey {
2   workspaceId: string;
3   versionId: string;
4 }
5 export type PromptTag = "draft" | "staging" | "production";

Section-specific depth (sec-01)

Anchor Problem Statement: Enterprise Prompt Management to prompt-management mechanics: explain how content-addressed blob store for bodies; postgres for metadata graph shifts resolver SLIs, how resolve api returns rendered text + fingerprint for cache keys changes storage economics, and how promotion lanes: draft → staging → production with approval gates shapes CI retry contracts—avoid generic "LLM platform" abstractions.

Why interviewers care

Prompt Management interviews reward crisp scope, explicit trade-offs, and failure stories—not generic microservice diagrams.

Interview checkpoint

Name one failure story for Problem Statement: Enterprise Prompt Management that proves you understand real outages, not happy-path diagrams.

Key Highlights

•Workspace → Project → Prompt template → immutable version commits
•Content-addressed blob store for bodies; Postgres for metadata graph
•Resolve API returns rendered text + fingerprint for cache keys
•Promotion lanes: draft → staging → production with approval gates

1	public record VersionKey(String workspaceId, String versionId) {}
2	public enum PromptTag { DRAFT, STAGING, PRODUCTION }

1	@dataclass(frozen=True)
2	class VersionKey:
3	workspace_id: str
4	version_id: str
5
6	class PromptTag(str, Enum):
7	DRAFT = "draft"
8	STAGING = "staging"
9	PRODUCTION = "production"

1	export interface VersionKey {
2	workspaceId: string;
3	versionId: string;
4	}
5	export type PromptTag = "draft" \| "staging" \| "production";