Design Genomics Data Pipeline

Hard45 min
1 / 30
understanding9 min read

Genomics pipeline framing: from sequencer to clinical report

How Genomics pipeline framing: from sequencer to clinical report (understanding) informs Genomics Data Pipeline architecture and interviewer depth.

Genomics pipeline framing: from sequencer to clinical report

Design a genomics data pipeline (Illumina / 23andMe / DNAnexus class): ingest terabase FASTQ from sequencers, run reproducible alignment and variant calling on HPC, and release DUO-gated CRAM/VCF to analysts. This section focuses on genomics pipeline framing: from sequencer to clinical report in the understanding phase—not a generic batch ETL lecture.

Numbers to state early

  • Metric A: 120 runs/day
  • Metric B: 30x WGS depth
  • Metric C: GxP audit trail

Mechanism

Instrument uploads demultiplexed FASTQ to a landing zone; LIMS binds sample_id, study cohort, and consent flags before any compute job is admitted.

Failure and edge cases

Treating genomics like a nightly ETL job ignores reproducible workflow versions, reference genome builds, and partial reruns from arbitrary pipeline stages.

When discussing Genomics pipeline framing: from sequencer to clinical report, trace Sequencer → LIMS → Orchestrator with 120 runs/day and explain how workflow pinning, reference builds, and consent/DUO gates differ from ad-hoc Spark jobs.

Java

javaOne Dark Pro
1public record SampleManifest(String sampleId, String studyId, String referenceBuild) {}

Python

pythonOne Dark Pro
1@dataclass(frozen=True)
2class SampleManifest:
3 sample_id: str
4 study_id: str
5 reference_build: str

TypeScript

typescriptOne Dark Pro
1export interface SampleManifest {
2 sampleId: string;
3 studyId: string;
4 referenceBuild: string;
5}

Why interviewers care

Genomics Data Pipeline interviews reward crisp scope, explicit trade-offs, and failure stories—not generic microservice diagrams.

Interview checkpoint

Name one failure story for Genomics pipeline framing: from sequencer to clinical report that proves you understand real outages, not happy-path diagrams.

Key Highlights

  • 120 runs/day
  • Sequencer → LIMS → Orchestrator
  • Instrument uploads demultiplexed FASTQ to a landing zone; LIMS binds sam
What interviewers want to hear
Lead Genomics pipeline framing: from sequencer to clinical report with 120 runs/day and explicit DUO + QC boundaries.
Pro tip
Pin WDL and reference hashes in sec-001; never tag workflows 'latest' in regulated runs.

Section Rescue Kit

Buzzwords to use:

Call CachegVCF Block

Safe statements:

  • "For Genomics pipeline framing: from sequencer to clinical report, I never release VCF without pinned WDL version and manifest hash in metadata."
  • "If HPC spot preempts, I resume from last _SUCCESS marker—not restart whole WGS from FASTQ unless QC failed."
Design Genomics Data Pipeline - System Design | WinJob | WinJob