nicu-validate 0.1.0

Federated validation client for NICU clinical decision support tools

nicu-validate

Federated validation for clinical decision support tools.

Lets NICU researchers validate an existing tool across multiple hospitals — without any raw patient data leaving any hospital. Each site runs a local evaluation script and returns only aggregate statistics (sensitivity, specificity, AUC). The coordinator pools results using DerSimonian-Laird random-effects meta-analysis and generates a publishable report.

Built for the BYU Technology Transfer Office.

---

The Problem

A NICU researcher builds a clinical decision support tool. It works on local data. To publish or get FDA clearance, they need multi-site validation. But HIPAA prevents raw patient data from leaving any hospital, and existing research networks (Vermont Oxford Network, CHNC) have 12–18 month approval timelines.

nicu-validate solves this in days, not months.

---

How It Works

Researcher                Site A (NICU)             Site B (NICU)
    |                          |                          |
    |-- registers study -----→ coordinator               |
    |                          |                          |
    |                  pip install nicu-validate          |
    |                  nicu-validate run --input data.csv |
    |                          |-- {sensitivity: 0.85,    |
    |                          |    specificity: 0.90,    |
    |                          |    n_patients: 80} ----→ coordinator
    |                                                     |
    |                                            (same flow)
    |                                                     |
    |←-- GET /studies/{id}/report ---------------------- coordinator
         D-L pooled sensitivity: 0.83 [0.76–0.90]
         D-L pooled specificity: 0.89 [0.83–0.94]
         Method: DerSimonian-Laird random-effects

No raw patient records leave any hospital. Ever.

---

Components

Coordinator Server (coordinator/)

FastAPI + SQLite server. Hosts the study registry, collects aggregate stats from sites, runs meta-analysis, and generates publishable reports.

Deploy:

cd coordinator
flyctl deploy   # or any Python host (Railway, Render, etc.)

Endpoints:

• POST /studies — create a validation study
• POST /studies/{id}/sites — register a participating NICU, receive API token
• POST /results — submit aggregate stats (site-side, authenticated)
• GET /studies/{id}/report — D-L meta-analysis report

Safety features:

• Small-cell suppression: sites with N<10 positives are flagged and excluded from meta-analysis (re-identification protection for rare conditions)
• IRB expiration tracking: coordinator blocks submissions from sites with lapsed IRB approval
• Append-only audit log for every submission (21 CFR Part 11 baseline)
• HTTPS enforced at the edge

Site Package (site-package/)

Zero-dependency pip package. Runs locally at each participating NICU. Never sends raw records — computes stats locally, POSTs only JSON aggregate stats.

Install:

pip install nicu-validate
nicu-validate init --coordinator https://your-coordinator.fly.dev --token <token>
nicu-validate run --input your_data.csv

Input CSV format:

condition,prediction
1,0.92
0,0.13
1,0.87
...

• condition: 0/1 ground truth (patient has the condition)
• prediction: 0/1 or continuous probability score (tool output)

---

Study Setup

python3 setup_study.py

Interactive script. Creates a study, registers participating sites, and prints the exact pip install command to send to each NICU.

---

HIPAA Compliance

Aggregate statistics (sensitivity, specificity, AUC) are not PHI under HIPAA Safe Harbor (45 CFR §164.514(b)). The nicu-validate package enforces small-cell suppression (N<10 positives suppressed) to prevent statistical re-identification of rare condition patients.

Note: For very rare conditions, consult a biostatistician to confirm the suppression threshold is appropriate for your specific condition's base rate before live site submissions.

---

Regulatory Notes

• Coordinator-level validation studies using aggregate-only outputs typically qualify as not-human-subjects research. Confirm with your institution's IRB.
• Each participating site handles their own local IRB authorization.
• If the underlying clinical decision support tool is subject to FDA 510(k) clearance, ensure the coordinator's audit log meets 21 CFR Part 11 requirements before using the validation data in a submission.

---

License

BYU Technology Transfer Office. Contact [e-mail] for licensing.