Design AI Chatbot - System Design | WinJob

Problem Statement: Enterprise RAG Support Chatbot

Design a multi-tenant customer-support chatbot that blends classical NLU (intent + slots) with retrieval-augmented generation (RAG) over a living knowledge base. Google, Rasa, and Microsoft interviews probe whether you treat the bot as an ML platform—not a thin wrapper around ChatGPT.

Personas

End customer expects grounded answers in <2.5s p99 with citations.
Support agent takes over when confidence < τ or policy triggers escalation.
ML engineer ships new embedding indexes and intent models weekly with canary promotion.

Core journeys

User asks billing question → intent billing_inquiry → hybrid retrieval → LLM answer with doc spans → optional human handoff.
Admin uploads PDF policy → async ingest → chunk + embed → searchable in <5 minutes p95.
Auditor exports redacted transcript with citation lineage for compliance.

Why this is hard

Hallucinations erode trust; stale KB answers wrong refunds; prompt injection via uploaded docs; token budgets explode on long threads; tenant isolation must hold in shared vector indexes.

Interview checkpoints (1/30)

Ground answers with citations or explicit refusal
Keep tenant_id on sessions, caches, and vector namespaces
State latency budget per stage (NLU, retrieve, rerank, LLM)

javaOne Dark Pro

1 // Problem Statement: Enterprise RAG Support Chatbot — design note 1
2 public record ChatMessage(String tenantId, String sessionId, String role, String text) {}

pythonOne Dark Pro

1 # Problem Statement: Enterprise RAG Support Chatbot — design note 1
2 @dataclass(frozen=True)
3 class ChatMessage:
4     tenant_id: str
5     session_id: str
6     role: str
7     text: str

typescriptOne Dark Pro

1 // Problem Statement: Enterprise RAG Support Chatbot — design note 1
2 export interface ChatMessage {
3   tenantId: string;
4   sessionId: string;
5   role: "user" | "assistant" | "system";
6   text: string;
7 }

Why interviewers care

AI Chatbot interviews reward crisp scope, explicit trade-offs, and failure stories—not generic microservice diagrams.

Interview checkpoint

Name one failure story for Problem Statement: Enterprise RAG Support Chatbot that proves you understand real outages, not happy-path diagrams.

Key Highlights

•Anchor Problem Statement: Enterprise RAG Support Chatbot to measurable chatbot SLOs
•Never omit tenant_id from caches or vector namespaces

1	// Problem Statement: Enterprise RAG Support Chatbot — design note 1
2	public record ChatMessage(String tenantId, String sessionId, String role, String text) {}

1	# Problem Statement: Enterprise RAG Support Chatbot — design note 1
2	@dataclass(frozen=True)
3	class ChatMessage:
4	tenant_id: str
5	session_id: str
6	role: str
7	text: str

1	// Problem Statement: Enterprise RAG Support Chatbot — design note 1
2	export interface ChatMessage {
3	tenantId: string;
4	sessionId: string;
5	role: "user" \| "assistant" \| "system";
6	text: string;
7	}