Design Vector Database - System Design | WinJob

Problem Statement: Billion-Scale Vector Search Platform

Design a production vector database for billion-scale embeddings with ANN search, metadata filtering, and multi-tenant isolation.

Why interviewers probe here

A vector database reads as "just an index" until you ask where the 768–1536D vectors live, how 50k+ aggregate QPS fan out across shards, and what a segment merge does to recall when it goes wrong. The interview wants the distributed-systems answer, not a single-node FAISS demo.

Operational detail

Frame two planes from the first sentence: a control plane for collections, index builds, and quotas, and a data plane for upserts and ANN queries. Pin the headline SLO — p99 < 100 ms for top-10 at 50k+ aggregate QPS — before drawing a single box.

Failure and edge cases

The failures that decide this question are recall collapse after a bad merge, hot shards when one collection goes viral, and just-upserted vectors that stay invisible until the segment flips. Each maps to a design choice you must defend.

javaOne Dark Pro

1 public record VectorId(String collection, String externalId) {}
2 public final class AnnQuery {
3   public float[] embedding;
4   public int topK = 10;
5   public Map<String, Object> filter = Map.of();
6 }

pythonOne Dark Pro

1 @dataclass(frozen=True)
2 class VectorId:
3     collection: str
4     external_id: str
5  
6 @dataclass
7 class AnnQuery:
8     embedding: list[float]
9     top_k: int = 10
10     filter: dict | None = None

typescriptOne Dark Pro

1 export interface VectorId { collection: string; externalId: string; }
2 export interface AnnQuery { embedding: number[]; topK?: number; filter?: Record<string, unknown>; }

Interview signal

Frame the vector DB as a distributed ANN system, not a single-node FAISS demo: an index type (HNSW vs IVF-PQ) chosen for the recall-vs-latency-vs-RAM trade-off, sharded with replicas, metadata filtering fused with ANN, at p99 < 100 ms and 50k+ QPS over billions of vectors. The tell is naming a real failure — recall collapse after a bad segment merge, or a hot shard on a viral collection.

Key Highlights

•Pinecone/Weaviate/Milvus-class ANN store for RAG and semantic search
•Collections with 768–1536D embeddings and JSON metadata filters
•Sub-100ms p99 top-10 queries at 50k+ QPS aggregate
•Streaming upserts with eventual index visibility

1	public record VectorId(String collection, String externalId) {}
2	public final class AnnQuery {
3	public float[] embedding;
4	public int topK = 10;
5	public Map<String, Object> filter = Map.of();
6	}

1	@dataclass(frozen=True)
2	class VectorId:
3	collection: str
4	external_id: str
5
6	@dataclass
7	class AnnQuery:
8	embedding: list[float]
9	top_k: int = 10
10	filter: dict \| None = None