Prodrome

Early warning for sepsis with a calibrated alert budget, validated on a hospital it never trained on.

A prodrome is the early set of signs that a disease is coming, before the specific symptoms that name it. Greek prodromos, "running before": a forerunner.

Not a medical device. Not cleared by any regulator. Not for clinical use. It predicts a recorded clinical suspicion, not a biological event. Read What this must never claim before quoting any number here.

The numbers

Trained on hospital system A, calibrated on held-out patients from A, then tested on every patient of hospital system B, which it never saw. External results first, because that is the number that means something.

utility is the published PhysioNet/CinC 2019 scoring function, where alerting through the whole beneficial window scores 1 and never alerting scores 0. detect is the share of septic patients ever alerted in time; in window is the stricter share alerted inside the window the utility score actually rewards. precision is the share of alerted patients who go on to become septic, against a base rate of 7.3 percent.

Three findings, and none of them flatter the model

A three-feature clock gets 98 percent of the utility score. The clock row alerts on nothing but how long the patient has been in the unit. No physiology, no laboratory values, nothing learned. It scores 0.2413 against the trained model's 0.2468. Septic stays in this corpus run 59 hours on average against 38 for the rest, so "this patient has been here a while" predicts "this patient will be recorded septic at some point", and the utility metric rewards it. Alerting on every patient from ICU hour 48 onward, with no model at all, scores +0.227.

What the trained model actually buys is visible in the other columns: it ranks far better (AUROC 0.802 against 0.628), catches more than twice as many patients inside the window that matters (0.153 against 0.065), and its probabilities mean something (calibration error 0.003 against 0.097, a factor of thirty). That is a real difference, and a table reporting utility alone would have hidden all of it. This is why the clock is a shipped configuration rather than a footnote.

The model is substantially reading the workup, not the patient. prodrome ablation refits the same model on restricted feature sets:

Measured values and bedside scores alone reach 47 percent of the full utility. The staleness and ordering pattern alone, with no measured value of any kind, reach 86 percent. Nobody orders a lactate for a patient they are relaxed about, so which tests exist encodes clinician suspicion, and the label is a record of clinician suspicion. That is a real signal and a useful one in a deployed system, but it is not an early warning: by the time the workup starts, somebody is already worried.

A guarantee calibrated at one hospital does not automatically survive the move to another. Each model states one sentence after calibration, and the report checks it on held-out patients:

The logistic model kept its promise at home and broke it at the other hospital, by a wide margin. Nothing about its internal report predicted that. This is the entire argument for treating external validation as the primary result rather than a robustness appendix.

And the two bedside scores could not make a promise at all. Neither qSOFA nor SIRS can certify 30 percent precision at any threshold within the alert budget, so their policy alerts on nothing and their utility is exactly zero. That is the honest output, not a failure of the harness.

Subgroup results on site B are stable rather than uniform: utility runs from 0.200 in the medical ICU and 0.202 for patients over 80, to 0.287 for ages 65 to 79 and 0.286 in the surgical ICU, with AUROC between 0.787 and 0.818. The full table is in reports/latest/default-siteB.json.

Quickstart

pip install "prodrome[dev]"
prodrome init            # writes configs/
prodrome data fetch      # 42 MB, 40,336 patients, no credentials needed
prodrome data prepare    # parses to columnar and pins the corpus digest
prodrome train           # fits on site A
prodrome calibrate       # chooses the threshold and states the promise
prodrome eval            # scores site A and site B, writes a report each
prodrome ablation        # is it reading the patient or the workup?

prodrome calibrate prints the sentence the system is committing to:

With probability at least 95% over the calibration draw, at most 4.0 alerts are raised
per patient-day and at least 30% of alerted patients go on to develop sepsis
(observed on the calibration slice: 0.17 alerts per patient-day, 34% precision).

Then make it worse on purpose and watch the gate refuse it:

prodrome eval --config configs/default.yaml --out reports/good.json
prodrome gate --baseline reports/good.json --candidate reports/degraded.json

roll back: default@A against default@A
- utility dropped by 0.1500 (baseline 0.3338, candidate 0.1838); the allowed drop is 0.0000.

That exit code is the point. Wire it into CI and a change that makes the system worse cannot be merged by accident.

The data

The PhysioNet/CinC 2019 challenge corpus, published under the Open Database License, which is why this project can exist in public: the models can be shared and every number can be reproduced by a stranger.

Nothing under data/ is committed or packaged. prodrome data fetch downloads it and prodrome data prepare verifies it, computing a digest over the sorted per-patient content hashes. Every report carries that digest and the gate refuses to compare two reports that disagree on it.

The label is SepsisLabel, which the organisers set six hours before a recorded clinical onset defined by suspicion of infection together with a rise in SOFA score. It is a proxy for a biological event, not the event.

How it works

hourly rows  ->  causal features  ->  model  ->  risk per hour
                       |                              |
              no feature sees the                alert policy
              future, proven per column         (threshold + promise)
                       |                              |
                       |                    above -> alert, with the guarantee
                       |                    below -> silence
                       v
        evaluate on the training site AND on a site never trained on
                       |
                  promotion gate  ->  promote or exit non-zero

Causality is tested, not asserted. Every feature at hour t is a function of hours 0 to t of that stay and nothing else. The test truncates each stay at seven different points, rebuilds, and compares every column: if one value changes, some feature read an hour that had not happened yet, and the test names the column. A second test rebuilds each stay alone and checks it against the same stay inside a crowd, which catches a window running off the start into the previous patient. Both were verified by injecting a deliberate leak and watching them fail.

The alert policy is a decision, not a score. On calibration patients the model has never seen, the threshold sweep takes the lowest cut whose precision, at its exact Clopper-Pearson lower bound, clears the floor, and whose alert rate stays inside the budget. When nothing qualifies, attainable is false, the threshold goes above every reachable risk, and the system alerts on nothing. A guarantee that cannot be certified must not be implied.

An alert is an event, not an hour. If the risk sits above the threshold for fourteen hours a ward sees one alarm, not fourteen. Alerts are counted as rising edges, which has a consequence worth knowing: the alert rate is not monotone in the threshold, because lowering it can merge two episodes into one.

Configurations

A gru sequence model and a sofa baseline are registered too; the recurrent one needs the torch extra. Every score here is partial where the corpus cannot support the published definition, and the names say so: this data carries no Glasgow Coma Scale, so qSOFA can never reach 3 and SOFA covers four of six organ systems.

Serve

pip install "prodrome[serve]"
prodrome serve --config configs/default.yaml

POST /score scores a whole stay. WS /stream takes one hour at a time and answers with the risk after that hour, which is the shape a real integration has and the one that demonstrates causality end to end: the service holds the history, appends the new hour, and has never been sent the next one. GET /health reports the loaded model, the corpus digest and the promise in force. GET /metrics exposes Prometheus counters. Every response carries the threshold, the guarantee and the disclaimer, because a clinical score arriving over an API with no context is how research code ends up somewhere it should not be.

Development

make install   # virtualenv with the dev extra
make check     # ruff, mypy strict, the offline test suite
make bench     # every configuration on both sites
make ablation  # what is the model reading?

The test suite runs entirely on a synthetic corpus this package generates, so a fresh clone with no download and no data licence still proves the pipeline works. CI runs lint, types and tests on Python 3.10 and 3.12, then the measured pipeline on synthetic data, then the gate.

What is committed: the code, the configurations, the reports under reports/latest/ that the gate compares against, and the leaderboard. What is not: anything under data/, any trained weight, any .env.

What this must never claim

• Not a medical device, not cleared by any regulator, not for clinical use.
• It predicts a recorded clinical suspicion, not a biological event. Every label in this field is a proxy and pretending otherwise is the standard dishonesty here.
• Two hospital systems in one country is not the world. The only supported claim is "it generalised from A to B by this much".
• The reported lead times are generous. A median of 33 hours largely reflects patients flagged early in a long stay, not a targeted prediction of that episode, which is why the strict in-window detection rate is reported beside it.
• The model leans heavily on the workup and the clock. See the ablation above.
• No survival or treatment-effect claim. This is a warning system, not a causal one.

Licence

Apache-2.0. The corpus is separately licensed under the Open Database License by PhysioNet and is neither redistributed nor committed here.