What I look for in a surveillance pipeline runbook
A practical checklist for writing runbooks that help laboratory and bioinformatics teams run surveillance workflows consistently.
A runbook is for the difficult day
A pipeline runbook is most valuable when the run does not go perfectly. Anyone can follow a few commands on a clean dataset. The real test is whether the document helps an analyst respond to missing FASTQs, failed controls, low coverage, database path errors, unexpected lineages, or a sample that passes one metric and fails another.
For surveillance teams, the runbook should reduce hesitation. It should tell the analyst what to check, what to rerun, when to stop, and when to escalate.
The minimum useful structure
- Purpose: the pathogen, sample type, sequencing platform, and scientific question.
- Inputs: file naming rules, metadata columns, references, databases, and config fields.
- Commands: environment creation, dry run, full run, report generation, and cleanup.
- Outputs: where to find consensus files, QC tables, trees, mutation reports, and logs.
- Decision rules: pass, review, fail, rerun, and limited-use categories.
Make judgment visible
Some decisions cannot be fully automated. A borderline Ct value, a partial genome, a contaminated negative control, or conflicting lineage results may require human judgment. The runbook should not hide those points. It should name them clearly and give the analyst a defensible path.
This is also useful for training. New analysts learn not only which commands to run, but why those commands matter and how evidence is weighed.
A good runbook leaves an audit trail
I like runbooks that instruct users to save software versions, command logs, configuration files, MultiQC reports, and final interpretation notes. When a result is later used in a manuscript, partner report, or dashboard, those artifacts make the analysis traceable.
Add an escalation matrix
A runbook becomes operational when it connects a trigger to an action, an owner, and a recorded artifact. For example, target-like reads in a negative control should lead to a defined contamination review; an unexpected lineage should lead to identity checks, reference review, and supervisor escalation; a database checksum mismatch should stop the run.
The matrix should distinguish a technical failure from a scientifically limited result. That prevents an analyst from silently rerunning until an output looks acceptable and makes it possible to audit why a sample was repeated, withheld, or reported with caveats.
- Trigger: the measurable condition that starts the review.
- Action: the immediate check, repeat, or containment step.
- Owner: the person or role responsible for the decision.
- Evidence: the log, QC table, ticket, or report note that records the outcome.
Operational details worth recording
In surveillance, small operational details become important later. Database versions, reference accessions, primer schemes, conda environment files, sample sheet revisions, and report templates should be preserved. When a result is questioned, these details are the difference between memory and evidence.
I also like runbooks that include a short troubleshooting section. Common failure modes such as missing FASTQ pairs, mismatched sample IDs, failed controls, low coverage, missing databases, and inconsistent metadata should have a clear first response.
- Record command logs and software versions.
- Store the exact config file used for each run.
- Define pass, review, limited-use, and fail rules.
- Include a troubleshooting section for common errors.