survaudit

Leave-one-cohort-out (LOCO) survival-signature benchmarking with built-in leakage audits — fully nested re-selection, permutation calibration of a forward search, and size-matched random-panel nulls — extracted from a nine-cohort breast-cancer gene-signature study (five-gene panel vs. PAM50, MammaPrint, Oncotype DX, the genomic grade index, and the Buffa hypoxia metagene).

The numerics (ridge-penalised Cox via Newton-Raphson, Breslow ties, numba-JIT; Harrell's concordance) are implemented from scratch and validated against scikit-survival; see survaudit.core.validate_against_sksurv.

Install

pip install survaudit
# figures need matplotlib, some sensitivity analyses need lifelines/statsmodels:
pip install "survaudit[all]"

Quickstart: harmonise your own cohort, then fit and evaluate

survaudit expects one expression matrix (genes x samples) and one clinical table (one row per sample) per cohort. harmonise_cohort converts these into the on-disk schema (<cohort>_expr.parquet, <cohort>_surv.parquet) that every other function reads, doing per-gene z-scoring within the cohort (never pooled across cohorts — this is what makes leave-one-cohort-out evaluation meaningful) and normalising follow-up time/event coding along the way.

import pandas as pd
from survaudit import harmonise_cohort, load_cohort, load_all, fit_ridge_cox, cindex

expr = pd.read_csv("my_expression.tsv", sep="\t", index_col="hugo_symbol")   # genes x samples
clin = pd.read_csv("my_clinical.tsv", sep="\t")                             # one row per sample

cfg = dict(
    sample_col="sample_id", time_col="os_months", event_col="os_event",
    endpoint="OS", platform="RNA-seq", scale="linear",     # "zscored" | "log2" | "linear"
    cov=dict(age="age", grade=None, size=None, node=None, stage=None,
             er=None, pr=None, subtype=None),               # None where you don't have it
)
qc_row = harmonise_cohort("MyCohort", cfg, expr, clin, out_dir="./harmonised")
print(qc_row)   # n, events, event_rate, median_fu_months, n_genes, join diagnostics

# Load it back (or several cohorts at once) and fit the same validated model:
store = load_all(["MyCohort"], data_dir="./harmonised")
X, t, ev, genes = store["MyCohort"]
panel = ["FLT3", "CLIC6", "SUSD3", "ZIC2", "P4HA2"]          # or your own gene list
beta = fit_ridge_cox(X[panel].values, t, ev, alpha=100.0)
risk = X[panel].values @ beta
print("concordance:", cindex(risk, t, ev))

If you have raw files on disk rather than loaded DataFrames, use harmonise_cohort_from_files(name, cfg, data_dir, out_dir) with cfg["expr"] /cfg["clin"] set to paths relative to data_dir (this is what harmonise_all does internally, looping a COHORTS registry — see survaudit.harmonise.COHORTS for a fully worked example of that registry, the nine cohorts this package was built from).

The audits, as a library or a CLI

Every analysis stage from the original study — fully nested re-selection, permutation calibration, size-matched nulls, the LOCO benchmark grid across four learners, decision-curve/likelihood-ratio incremental-value tests, figure and table generation — ships as one importable, side-effect-free module under survaudit.pipeline, runnable exactly as it was written (reads its inputs and writes its outputs relative to the current directory):

survaudit --list                      # show every available stage
survaudit harmonise                   # OUT=./harmonised python -m survaudit.pipeline...
survaudit run_nested_selection        # writes nested_selection_*.csv/.json here
survaudit run_permutation_search 1000 0
survaudit make_tables                 # regenerate paper/tabN_*.tex from results/

equivalently python -m survaudit.pipeline.run_nested_selection. Each module's docstring documents what it reads and writes; several were written for a specific nine-cohort study and assume gene sets / cohort names from that study by default (survaudit.core.NOVEL5, OS6, SEC3) — override those where the stage exposes them as arguments, or adapt the module for your own gene sets (they are ordinary, short, readable Python scripts).

What's validated, what's a starting point

survaudit.core (the Cox fitting, concordance, and the fully-nested re-selection / candidate-pool / forward-search primitives) and survaudit.harmonise (cohort ingestion) are the reusable engine and are covered by validate_against_sksurv plus a byte-identical round-trip check against the original study's nine harmonised cohorts. survaudit.pipeline.* are faithful extractions of that study's specific analysis scripts — useful as a worked, runnable reference for how to wire the engine into a leakage audit on your own data, but written for that study's cohort names, gene sets, and directory layout, not yet generalised into a parametrised API.

License

MIT (see LICENSE).