Design Deployment Verification

Medium40 min
1 / 30
understanding6 min read

Problem Statement: Post-Deploy Verification

How Problem Statement: Post-Deploy Verification (understanding) informs Deployment Verification architecture and interviewer depth.

Design Deployment Verification

A deployment verification platform proves that a newly deployed artifact is safe to receive production traffic before the release is marked successful. Harness Continuous Verification, Spinnaker pipeline gates, and LaunchDarkly-style observability hooks all treat verification as a first-class stage—not an afterthought once users already hit regressions.

Why interviewers ask this

  • Separates health checks (process up) from correctness checks (business paths work).
  • Tests whether you can fail closed when telemetry is missing.
  • Surfaces integration with CD orchestrators, metrics, and automated rollback.

Core journeys

Release engineer: deploy → watch verification dashboard → promote or rollback.

SRE: define SLI policies, tune soak windows, investigate false positives.

Service owner: register smoke tests and dependency probes per microservice.

Success criteria

  • Verification completes in < 15 minutes for tier-1 services.
  • False rollback rate < 2% after policy tuning.
  • Every decision auditable with artifact digest + metric snapshot IDs.
javaOne Dark Pro
1public final class VerificationGateEvaluator {
2 public GateVerdict evaluate(SliSnapshot candidate, SliSnapshot baseline, GatePolicy policy) {
3 if (candidate.sampleCount() < policy.minSamples()) {
4 return GateVerdict.failClosed("insufficient_samples");
5 }
6 if (candidate.p99LatencyMs() > baseline.p99LatencyMs() * policy.maxLatencyFactor()) {
7 return GateVerdict.rollback("latency_regression");
8 }
9 if (candidate.errorRate() - baseline.errorRate() > policy.maxErrorDelta()) {
10 return GateVerdict.rollback("error_budget_burn");
11 }
12 return GateVerdict.pass();
13 }
14}
pythonOne Dark Pro
1async def run_smoke_suite(deployment_id: str, base_url: str, cases: list[dict]) -> dict:
2 results = []
3 for case in cases:
4 resp = await http_request(
5 method=case["method"],
6 url=f"{base_url}{case['path']}",
7 headers={"X-Deployment-Id": deployment_id},
8 timeout_s=case.get("timeout", 5),
9 )
10 ok = resp.status < 400 and case["assert"](resp.body)
11 results.append({"name": case["name"], "ok": ok, "status": resp.status})
12 failed = [r for r in results if not r["ok"]]
13 return {"passed": len(failed) == 0, "failed": failed, "results": results}
typescriptOne Dark Pro
1export type VerificationPhase = "pre-traffic" | "post-traffic" | "soak";
2
3export interface VerificationRunRequest {
4 deploymentId: string;
5 artifactDigest: string;
6 environment: string;
7 phases: VerificationPhase[];
8}
9
10export async function startVerificationRun(
11 req: VerificationRunRequest,
12): Promise<{ runId: string; status: "queued" }> {
13 return post("/api/v1/verification/runs", req, {
14 headers: { "Idempotency-Key": req.deploymentId },
15 });
16}

How to open this one

The framing that lands for post-deploy verification is automated, gated proof that a release is healthy before it gets full traffic: smoke tests, SLO checks, and canary analysis that can halt and roll back without a human. Lead with treating absence of signal as failure — no metrics means not healthy, so do not promote — and the failure story that proves it: a deploy passes its own health check but quietly burns the error budget, and automated rollback catches what the green checkmark missed. That shows you understand verification gates promotion on evidence, not on the deploy merely succeeding.

Key Highlights

  • Fail closed when observability or smoke evidence is incomplete.
  • Pin every verification run to the immutable artifact digest.
  • Separate pre-traffic, post-traffic, and soak phases in the gate.
  • Integrate pass/fail/hold webhooks with Spinnaker, Harness, or GitHub Actions.
State assumptions
Quantify verification duration and rollback SLO before drawing boxes.
Fail open trap
Passing deploys without metrics is how customer-impacting regressions slip through.

Section Rescue Kit

Buzzwords to use:

Continuous VerificationFail Closed

Safe statements:

  • "Let me separate deploy-time checks from steady-state synthetic monitoring."
  • "Verification must bind to the immutable artifact digest, not only the service name."
Design Deployment Verification - System Design | WinJob | WinJob