Design a Docker to ECS or EKS Migration

Medium45 min
1 / 30
System design answer

Design a Docker to ECS or EKS Migration

Medium45 min30 sections
understanding•8 min read

Problem Statement: Replatforming Live Docker Fleets Without a Downtime Incident

Frames the migration as a replatforming program with four planes, not a greenfield container project.

Problem statement

Design a migration platform that moves an existing fleet of Docker-based services off manually managed VMs (docker run scripts and docker-compose hosts on EC2) onto AWS ECS (Fargate or EC2 launch type) or EKS, with near-zero customer-facing downtime, full transfer of configuration and secrets, and a reversible traffic cutover at every step.

The estate we assume: 40 services, roughly 120 containers in production, running on 25 manually provisioned EC2 instances. Deployments are shell scripts that SSH into hosts, docker pull, and docker restart. There is no orchestrator, no service discovery beyond hardcoded hostnames and a shared HAProxy, secrets live in .env files copied by hand, and scaling means adding VMs and editing load balancer config. This is a real and common estate: teams containerized early to escape VM dependency hell, then stopped before orchestration.

Why this is not a greenfield design

A from-scratch system design asks what to build. A migration design asks what must stay true while two worlds run in parallel. Three properties dominate:

  1. The running system is the source of truth. Undocumented environment variables, implicit host networking assumptions, local disk writes, and cron jobs on the same VM all exist. Discovery precedes design.
  2. Two environments must coexist. Old VMs and the new cluster share databases, caches, queues, and DNS. The migration layer must route traffic between them and move state safely.
  3. Every step must be reversible. The rollback path is designed before the forward path. If the new cluster misbehaves at 25 percent traffic, traffic returns to the VMs in under five minutes without redeploying anything.

This is the R in AWS's six R migration model: replatform. The application code barely changes; the runtime substrate, networking, identity, config, and delivery pipeline change completely. AWS launched ECS in November 2014, Fargate in November 2017, and EKS reached general availability in June 2018, so the target platforms are mature; the risk lives in the transition, not the destination.

The four architectural planes

  1. Build and registry plane: image standardization, base image policy, ECR repositories, scanning gates, and promotion across environments.
  2. Control plane: clusters, task definitions or Kubernetes manifests, capacity providers or node groups, and the deployment orchestrator.
  3. Traffic plane: ALB weighted target groups, DNS, session stickiness, dual-run and shadow traffic, and cutover automation.
  4. Governance plane: the migration ledger, per-service wave state, rollback budgets, cost telemetry, and audit evidence.

A strong answer keeps these planes separate. The traffic plane must work while the control plane is being rebuilt, and the governance plane must be able to freeze a wave even mid-shift. The cloud coordinates and records; the load balancer and health checks decide whether a cutover proceeds.

Key Highlights

  • •Migration is replatforming: code stays, substrate, networking, identity, config, and pipeline change completely.
  • •The running estate is the source of truth; discovery of implicit behavior precedes any cluster design.
  • •Old VMs and the new cluster coexist and share data stores, so dual-run routing is a first-class component.
  • •Rollback is designed first: traffic returns to VMs in under five minutes at any shift percentage.
  • •Four planes: build/registry, control, traffic, governance — each can degrade without corrupting the others.
Lead With Reversibility
State in the first two minutes that every cutover step is reversible within five minutes and that rollback is designed before the forward path. That single sentence separates migration engineers from greenfield designers.
Do Not Start With ECS Versus EKS
Choosing the orchestrator before inventorying state, secrets, and network coupling produces a cluster that cannot run the estate. Discovery first, platform choice second.

Section Rescue Kit

Buzzwords to use:

ReplatformingDual-Run

Safe statements:

  • "Before picking ECS or EKS, let me enumerate what the current fleet implicitly depends on."
  • "I will separate what must change, what must stay identical, and what we can only discover by running both systems."
Design a Docker to ECS or EKS Migration - System Design | WinJob | WinJob