Preflight cbendercheck dev

[adamfreedman]

Summary

Adds preflight quality/integrity gates and an adaptive CellBender re-run to the preprocessing workflow. The goal is to catch unusable inputs before the DAG launches, and to automatically react to a poor CellBender training curve rather than silently keeping a bad fit. Three self-contained additions, each with tests; no changes to existing resource requests or default behavior for healthy samples.

1. Preflight cell-count gate

A parse-time check counts called cells in each sample's CellRanger filtered_feature_bc_matrix (barcodes line count — no matrix load) and handles samples with too few cells to cluster before any job launches, governed by a new preflight_mode:

• skip (default) — drop the sample from the DAG and report it; re-checked every run, so a re-sequenced sample with more cells is picked up automatically
• error — hard-stop the run · warn — report only · off — disable

Only the CellRanger filtered count is knowable preflight (emptyDrops / cellbender_fromraw call cells at runtime), so it gates on that as the whole-sample viability signal. The runtime min-cells guard and the results/low_quality_samples/ quarantine remain the backstop for the other arms.

2. CellBender raw-matrix preflight

For the cellbender_fromraw arm, before any job launches (reads only the MTX header):

• Hard error on a raw matrix.mtx that is unreadable, empty/degenerate (zero features/barcodes/nonzeros), or has fewer barcodes than the sample's filtered matrix (raw must be a superset) — a broken/wrong input to fix or exclude.
• Warning (non-blocking) when raw droplets are < 2× the CellRanger-filtered cell count — too few empties for CellBender's ambient estimate, often a sign a filtered matrix was placed on the raw path.

These are input-integrity checks, so unlike the cell-count gate they are not governed by preflight_mode. Runs only when cellbender_fromraw is configured.

3. Adaptive CellBender re-run

After the initial CellBender run, the HTML report's automated assessment is parsed:

• "learning curve looks normal" → keep the initial run.
• "re-run with half the current learning rate" → if cellbender_adaptive_rerun is on, re-run at half cellbender_learning_rate and keep the re-run only if its curve is normal; otherwise fall back to the initial run.
• Any unrecognized assessment → hard error, so report-parsing edge cases surface rather than passing silently.

Both runs' reports are archived (_report_initial.html / _report_rerun.html) and a per-sample status file records the outcome. A new cellbender_adaptive_summary rule aggregates every sample into results/cellbender/cellbender_adaptive_summary.tsv, reporting which samples needed a re-run and whether it resolved the learning curve.

New config keys (validated, with documented defaults)

preflight_min_cells: 100
preflight_mode: "skip"             # skip | error | warn | off
cellbender_learning_rate: 0.0001   # CellBender's default; re-run uses exactly half
cellbender_adaptive_rerun: true    # false = always keep the single initial run

New outputs

• results/cellbender/cellbender_{sample}_report.html + archived _report_initial.html / _report_rerun.html
• results/cellbender/cellbender_{sample}_adaptive_status.txt (per-sample outcome)
• results/cellbender/cellbender_adaptive_summary.tsv (run-level rollup)

Tests

New/updated coverage (+965 / −44 across 12 files): assess_report unit tests (normal / rerun / unknown), every adaptive orchestration branch via a fake CellBender, the run-level summary, the raw-matrix preflight (4 cases + minimal-datadir fixture), the cell-count gate (skip/error/warn/off/pass), and the new config-validation cases.

Compatibility

No change to the CellBender SLURM request (48 h GPU walltime, runtime = 2880) versus main — the adaptive wrapper runs both passes inside the same job, so the existing walltime covers a re-run. Behavior for healthy samples is unchanged; all new gates either no-op (preflight_mode: skip only affects low-cell samples) or default to CellBender's own settings.