Design a Git Multi-Repo CI/CD

Medium45 min
1 / 31
System design answer

Design a Git Multi-Repo CI/CD

Medium45 min31 sections
understanding•10 min read

Problem Statement: Coordination Is the Product, Not Builds

Frames multi-repo CI/CD as a graph-coordination and consistency problem, not a collection of independent build boxes.

Problem statement

Design a CI/CD orchestrator for an engineering organization whose code lives in many Git repositories. Repos depend on each other: service A imports library B, mobile app C consumes SDK D, and services E and F share proto contract G. When someone merges a change to B, the system must detect it, compute which downstream repos are affected, rebuild and retest the right set, publish versioned artifacts, and keep the whole estate in a coherent, deployable state. When many teams merge at once, the system must decide which combinations of commits are allowed to land together without breaking anyone else.

Single-repo CI is a solved-ish problem: one webhook, one pipeline, one artifact. Multi-repo CI is a distributed coordination problem wearing a DevOps costume. The hard questions are not "can we run make?" but: which builds are affected by this commit, which combination of repository SHAs is mutually consistent, which merges may proceed in parallel without breaking the trunk of any repo, and how do we cache enough work that the graph does not rebuild the world on every push.

Why this is distinctive

A wrong build in a single repo delays one team. A wrong decision in a multi-repo orchestrator can poison shared artifacts, break every consumer of a library, or let incompatible versions reach production together. The design therefore separates three concerns. Change detection is an ingestion and identity problem: webhooks arrive at-least-once, out of order, sometimes duplicated, and must be normalized into canonical commit events. Graph reasoning is a data problem: the dependency graph is versioned per manifest revision, and impact analysis must be computed against the graph as it existed at a given commit, not today's graph. Execution is a scheduling and correctness problem: builds run in topological waves, reuse cached artifacts by content hash, and gate merges through a serialized queue when the organization demands always-green trunks.

Google's public monorepo paper reports roughly two billion lines of code, nine million files, and about 60,000 commits per week served by one Piper repository and Blaze builds. Google largely eliminates this problem by removing the multi-repo boundary. Most companies cannot. Netflix publicly describes running over a thousand microservices, each in its own repository, coordinated by Spinnaker pipelines and shared tooling. That is the reality this design serves.

The four architectural planes

  1. Ingestion plane: webhook receivers, signature verification, event normalization, commit canonicalization.
  2. Graph plane: manifest parsing, versioned dependency graph, impact analysis, cycle detection.
  3. Orchestration plane: pipeline DAG execution, merge queue, release trains, priorities and quotas.
  4. Execution plane: agent pools, hermetic builds, remote artifact cache, artifact store, logs and evidence.

A strong answer keeps these planes separate. The graph plane can be wrong without builds running. The orchestration plane can be degraded without losing the ability to answer "what depends on what." And the execution plane must remain safe even when every control-plane component is briefly unreachable.

Key Highlights

  • •The core product is cross-repo coordination: impact analysis, version alignment, and merge gating.
  • •Dependency graphs are versioned per manifest revision; impact is computed at a commit, not against live state.
  • •Google's monorepo (roughly 2B lines, 60K commits/week per Potvin and Leiserson) avoids this problem; multi-repo orgs must engineer it.
  • •Four planes: ingestion, graph, orchestration, execution.
  • •A wrong orchestrator decision can poison shared artifacts across every consumer repo.
Lead With Coordination
State in the first two minutes that the hard problem is cross-repo consistency and impact analysis, not running builds. That instantly separates a multi-repo design from a single-repo CI tutorial.
Do Not Draw N Independent Pipelines
A design where each repo runs its own pipeline with no graph, no shared cache, and no merge gate ignores the entire question. The interviewer will push on 'what happens when library B breaks service A?'

Section Rescue Kit

Buzzwords to use:

Impact AnalysisVersioned Dependency Graph

Safe statements:

  • "I will separate change detection, graph reasoning, orchestration, and execution before choosing any storage."
  • "The interesting failures here are coordination failures, so let me define who owns version alignment first."
Design a Git Multi-Repo CI/CD - System Design | WinJob | WinJob