Design a Bastion Approach for Deployments

Medium45 min
1 / 30
System design answer

Design a Bastion Approach for Deployments

Medium45 min30 sections
understanding•11 min read

Problem Statement: The Bastion as the Only Front Door to Production

Frames the bastion not as a jump host but as an identity-aware access, credential, and audit platform for every human and machine path into production.

Problem statement

Design a bastion approach for deployments in which every interactive access and every deployment connection into production passes through a dedicated, identity-aware gateway. No developer laptop, no CI runner, and no vendor tool may open a direct path to a production host. The platform must authenticate the actor, evaluate a policy, mint a short-lived credential scoped to exactly the targets and commands that were approved, broker or record the session, and write every action into a tamper-evident audit trail.

The naive version of this design is a single jump host with a shared SSH key in authorized_keys. That pattern fails a serious interview immediately: one static secret shared by everyone means no attribution, no revocation, no session evidence, and one compromised laptop becomes full production access. The design we will build instead treats the bastion as four cooperating planes.

The four planes

  1. Identity and grants plane. Decides WHO may access WHAT, WHEN, and WHY. Integrates with the corporate identity provider, accepts just-in-time access requests with reasons and ticket links, enforces approvals, and issues grants with explicit TTLs.
  2. Credential issuance plane. Turns an approved grant into a short-lived, narrowly scoped credential: an SSH user certificate signed by an internal certificate authority, an OIDC token for a CI workload, or a one-time session ticket. Static keys are eliminated, not managed.
  3. Session plane. Brokers the actual connection: an SSH proxy that terminates the client connection, authorizes it against the live grant, forwards to the target, records the byte stream and the parsed command stream, and can inject live review or forced termination.
  4. Audit and governance plane. Collects every decision, credential mint, session event, and command into an append-only, hash-chained log with WORM retention, and feeds access reviews, incident forensics, and compliance evidence.

Why deployments specifically

The brief couples human access with deployment connections, and that coupling is the interesting part. A modern fleet of tens of thousands of hosts is deployed by pipelines that fan out to hundreds of hosts concurrently. If deploy agents carry long-lived keys, the blast radius of one compromised runner equals the entire fleet. The bastion approach therefore issues per-pipeline, per-run, per-environment credentials that expire in minutes, bind them to the exact change ID, and record which change executed which command on which host. Deployment provenance becomes a queryable fact instead of a tribal memory.

Deployment success versus security invariants

This system has two different correctness models and a strong answer keeps them separate. Access convenience is an eventually-available workflow: request, approve, issue, connect, close. Security invariants are not negotiable: no connection without a live grant, no credential with unbounded lifetime, no session without audit capture, no CA signing key without hardware protection, and no audit record that can be silently edited. When the platform degrades, convenience degrades first; the invariants fail closed.

Public evidence that the pattern is real

Netflix open-sourced BLESS, a certificate authority that issues short-lived SSH certificates and removed the need for shared bastion keys; Meta publicly described migrating its fleet access from static authorized_keys files to short-lived SSH certificates; HashiCorp Vault's SSH secrets engine issues signed certificates with TTLs and principal restrictions and is used by large production fleets; and managed alternatives such as AWS Systems Manager Session Manager show the brokered-session pattern without any self-managed jump host at all. These are cited industry signals; every scale number used for capacity planning in this answer is an explicit design assumption stated where it appears.

What we will not design

We will not design the operating system hardening of individual targets, the full CI/CD artifact pipeline, network segmentation in depth, or a general secrets manager. We design the contracts, state machines, failure semantics, and evidence flows around those dependencies.

Key Highlights

  • •A bastion is four planes: identity and grants, credential issuance, session brokering, and audit governance.
  • •Static shared keys are replaced by short-lived SSH certificates and scoped workload tokens.
  • •Deployment connections receive per-run credentials bound to a change ID, not long-lived agent keys.
  • •Convenience is an eventually-available workflow; security invariants fail closed and never degrade.
  • •Netflix BLESS, Meta certificate SSH, Vault SSH engine, and AWS Session Manager prove the pattern in production.
Lead With the Front Door
State in the first two minutes that the design removes every direct path into production: targets stop trusting user keys entirely and trust only the internal certificate authority. That single sentence separates a platform design from a jump-host sketch.
Do Not Design a Single Box
A design in which one VM holds all keys and all sessions has a single point of compromise and a single point of failure. Separate grants, issuance, sessions, and audit into independently scalable and independently auditable services.

Section Rescue Kit

Buzzwords to use:

Short-Lived SSH CertificateFail-Closed Invariant

Safe statements:

  • "I will separate the four planes first, because each has a different consistency model and a different failure behavior."
  • "Before choosing components, let me define which decisions must fail closed and which may degrade gracefully."
Design a Bastion Approach for Deployments - System Design | WinJob | WinJob