kreview

Advanced cfDNA Fragmentomics Core Evaluation Engine

🧬 Overview

kreview is a production-grade, notebook-first (nbdev) evaluation engine designed for high-throughput cancer liquid biopsy fragmentomics feature analysis. Developed at Memorial Sloan Kettering (MSKCC), it processes cohorts containing tens of thousands of samples using an embedded DuckDB query engine with chunked I/O and automatic retry logic.

📖 Full Documentation

🚀 Features

• 5-Tier ctDNA Taxonomy: MSK-IMPACT paired-inference to label True ctDNA+, Possible ctDNA+, Possible ctDNA−, Healthy Normal, and Insufficient Data. Optional CH hotspot demotion via --ch-hotspot-maf.
• DuckDB Query Engine: In-memory read_parquet bindings with chunked I/O and exponential backoff retry for cohort-scale feature loading.
• Multi-Model Evaluation: Logistic Regression, Random Forest, and XGBoost (CPU) plus TabPFN and TabICL (GPU) with Stratified K-Fold CV, SHAP explainability, and subgroup analysis.
• Nested CV Feature Ablation: Automated feature group subset selection via inner-loop cross-validation, eliminating non-informative feature groups before final evaluation. Uses sensitivity_at_100spec_healthy as the optimization metric.
• Feature Selection: mRMR (Minimum Redundancy Maximum Relevance) as default strategy — iteratively selects features maximizing target relevance while minimizing inter-feature redundancy. Legacy hybrid_union (AUC ∪ MI) also available.
• Multimodal Stacking: Cross-evaluator fusion via super-matrix with Mutual Information or Boruta-SHAP selection, followed by stacking ensemble + ablation analysis.
• Interactive Dashboards: Plotly-native HTML reports with ROC curves, violin plots, SHAP beeswarm/waterfall, mRMR scatter plots, per-cancer-type sensitivity tables, and Decision Curve Analysis.
• Nextflow HPC Integration: Decomposed multistage DAG for SLURM-based HPC execution with per-evaluator parallelism, GPU scheduling, and automatic retry logic.
• 26 Built-In Evaluators: Modular extractors covering fragment sizes (FSC, FSD, FSR), nucleosome protection (WPS, TFBS), cleavage motifs (EndMotif, BreakPointMotif), chromatin accessibility (ATAC), motif divergence (MDS), and orientation (OCF).

🏗️ Pipeline Architecture

graph LR
    A[Label] --> B["Extract ×N"]
    B --> C[Select]
    C --> D["Ablate (opt)"]
    D --> E["Eval CPU"]
    D --> F["Eval GPU"]
    C --> E
    C --> F
    C --> G[Fuse]
    E --> H[Scoreboard]
    F --> H
    E --> I["Eval Multimodal"]
    F --> I
    G --> I
    H --> J[Report]
    I --> K["Report Multimodal"]

The pipeline runs as a Nextflow multistage DAG — one implementation, scattered per-evaluator. Use -profile docker locally and -profile iris/slurm on HPC. Supported Nextflow: v25–v26.

⚙️ Quick Start

Installation

[!IMPORTANT] Quarto is required for programmatic dashboard generation. kreview declares quarto-cli as a core dependency, so pip install kreview provides the Quarto executable automatically (and the Docker images ship it). If the pip-provided binary misbehaves in your environment, install Quarto from your OS package manager as a fallback (see the Quarto guide).

Option 1: Docker (Recommended "Batteries-Included" Method)

The easiest way to run kreview without managing external dependencies is to use our pre-built Docker containers (hosted on GHCR). They ship with Python 3.12, all ML libraries, and quarto:

# CPU image (~1.5 GB) — for all standard pipeline processes
docker pull ghcr.io/msk-access/kreview:latest

# GPU image (~8-10 GB) — adds PyTorch, TabPFN, TabICL (requires NVIDIA drivers)
docker pull ghcr.io/msk-access/kreview:latest-gpu

# The images are driven by Nextflow, one container per pipeline stage:
nextflow run /path/to/kreview/nextflow/main.nf -profile docker --outdir results/ ...

# Individual stages can also be invoked directly for debugging:
docker run -v /your/data:/data ghcr.io/msk-access/kreview:latest \
  label --cancer-samplesheet /data/cancer.csv ...

Option 2: Local Install (Pip)

quarto-cli is a declared dependency, so a plain pip install provides Quarto. Only if the pip-provided binary misbehaves, install Quarto from your OS manager (e.g. brew install quarto) as a fallback.

git clone https://github.com/msk-access/kreview.git
cd kreview
pip install -e .            # CPU models only
pip install -e ".[all]"     # + arfs feature selection, docs, dev, test (CPU)
pip install -e ".[gpu]"     # + TabPFN, TabICL (requires CUDA)

Running the Pipeline

Local (single machine, Docker)

nextflow run /path/to/kreview/nextflow/main.nf \
  --cancer_samplesheet "/path/to/cancer/samplesheet.csv" \
  --healthy_xs1_samplesheet "/path/to/healthy/xs1/samplesheet.csv" \
  --healthy_xs2_samplesheet "/path/to/healthy/xs2/samplesheet.csv" \
  --cbioportal_dir "/path/to/cBioPortal_MAF_CNA_SV/" \
  --krewlyzer_dir "/path/to/unified_krewlyzer_results" \
  --outdir output/ \
  --strategy mrmr \
  --top_percentile 10 \
  --ch_hotspot_maf "/path/to/ch_hotspots.maf" \
  -profile docker

Individual stages are also available as subcommands (kreview label, extract, select, eval cpu|gpu, fuse, report) for debugging a single step outside the DAG.

HPC (Nextflow + SLURM)

nextflow run /path/to/kreview/nextflow/main.nf \
  --cancer_samplesheet /path/to/cancer.csv \
  --healthy_xs1_samplesheet /path/to/healthy_xs1.csv \
  --healthy_xs2_samplesheet /path/to/healthy_xs2.csv \
  --cbioportal_dir /path/to/cbioportal/ \
  --krewlyzer_dir /path/to/manifest.txt \
  --outdir /path/to/output/ \
  --run_gpu_eval true \
  --gpu_models "tabpfn,tabicl" \
  --run_ablation true \
  --run_multimodal_eval true \
  -profile iris

Dashboard Access

Once finished, open the generated HTML reports:

open output/reports/ATAC_dashboard.html

🧪 Feature Selection

See Statistical Evaluation for full documentation.

📓 nbdev Architecture

This project operates as an nbdev repo. Do not edit .py scripts manually in kreview/. Build natively inside Jupyter notebooks within nbs/ and trigger:

nbdev-export && black kreview/   # note: `python3 -m nbdev.export` is a silent no-op

📚 Resources

• Documentation — Full user and developer guide
• Contributing — How to contribute
• Changelog — Version history