Clinical Site Statistics Rollup & Predictive Modeling Pipeline

Lewis Katz School of Medicine at Temple University Center for Biostatistics & Epidemiology 2026-08-04

Project Overview

This repository contains the multi-center pipeline, statistical models, and manuscript deliverables consolidating longitudinal Electronic Health Record (EHR) inpatient data across five clinical health systems: - TEMPLE (Lewis Katz School of Medicine at Temple University) - PSU (Penn State University) - Hopkins (Johns Hopkins University) - PITT (University of Pittsburgh Medical Center) - Geisinger (Geisinger Health System)

The pipeline standardizes encounter types and procedure coding, executes longitudinal lookback window aggregations (1-year and 2-year), maps lab values via LOINC codes, generates Table 1 descriptive statistics rollups, and trains patient-level traditional machine learning models (Logistic Regression, Random Forest, HistGBM, and XGBoost) using Stratified 5-Fold Cross-Validation on $N = 347,541$ qualifying patient encounters.

Repository Structure & Extract Downloads Directory

To maintain a clean working repository, server extracts and download archives are organized under the extracts/ directory:

kyle_out_zip/
├── extracts/
│   ├── from_server/          # Drop newly downloaded server ZIP archives here (*.zip)
│   └── archive/              # Archived historic server extracts and legacy datasets
├── old_logs/                 # Archived pipeline execution logs and raw run dumps
├── Dr_Rubin_Manuscript_Deliverables_20260729/   # Formatted manuscript tables & figures
├── Dr_Rubin_Manuscript_Deliverables_20260729.zip
├── eDERRI_Traditional_ML_Models_Manuscript.qmd  # Quarto manuscript draft (v1.11.6 with DALEX & Appendices A & B)
├── tasks/add/appendix_include/                   # Biostatistical & metric appendix modules
├── generate_manuscript_tables.py
├── generate_shap_4panel.py
├── generate_dalex_4panel.py
├── generate_shap_blowup_analysis.py
├── build_dr_rubin_deliverables.py
├── diagnose_lr_utilization.py
├── pyproject.toml
├── test_table1.py
├── test_pipeline_and_paths.py
└── test_manuscript_data_fixes.py

All Python helper scripts (generate_manuscript_tables.py, generate_shap_4panel.py, generate_dalex_4panel.py, build_dr_rubin_deliverables.py, diagnose_lr_utilization.py) automatically search extracts/from_server/ and extracts/ recursively to detect and process the latest extract ZIP files.

Deliverables & Manuscript Table Specifications

1. Formatted Manuscript Tables (Dr_Rubin_Manuscript_Deliverables_20260729/01_Manuscript_Tables/)

• Manuscript_Tables_Combined.xlsx: Holds sheets matching Dr. Rubin’s exact shells:
  • Table 1. Descriptive: Baseline characteristics for all 83 features (Whole cohort, Readmission subset, Non-readmission subset).
  • Table 2. LogReg: Multivariable Logistic Regression risk factors for all 93 features (Dual Unscaled Marginal ORs & Normalized ORs, 95% CIs, P-values).
  • Table 3. Model performance: Model discrimination and calibration measures (AUC, F1, Sensitivity, Specificity, Brier Score across model types; all cross-validation $\text{SD} < 0.01$).
• Manuscript_Tables_By_Site.xlsx: Per-site Table 1s, Table 2s, and master site performance metrics.

2. Publication Figures (Dr_Rubin_Manuscript_Deliverables_20260729/02_Manuscript_Figures/)

• Figure_Top10_DALEX_4Panel.png: High-resolution 300 DPI 4-panel summary plot comparing top 10 feature importances via DALEX variable dropout loss across models.
• Figure_Top10_SHAP_4Panel.png: High-resolution 300 DPI summary plot comparing top 10 features via SHAP Shapley values across models.
• Figure_Prior_Utilization_SHAP_Blowup.png: 4-panel architectural comparison for the top utilization feature (prior_IP_OS_ED_count) showing how LR (linear tail), RF (bimodal butterfly), HistGBM (smooth saturation), and XGBoost (regularized step) process risk differently.
• Figure_SHAP_4Panel.png: Combined 4-panel full SHAP summary figure.
• Figure_HistGBM_SHAP.png: Standalone 300 DPI HistGBM SHAP summary plot.

Empirical Diagnostic Findings

A 2-feature Logistic Regression diagnostic experiment (diagnose_lr_utilization.py) was conducted on the full combined dataset ($N = 347,541$ encounters): - Full 93-Feature Model: $\text{AUC-ROC} = 0.787$, $\text{Brier Score} = 0.128$ - 2-Feature Utilization Sub-Model (prior_IP_OS_ED_count + 12-month readmissions): $\text{AUC-ROC} = 0.761$, $\text{Brier Score} = 0.136$ - Finding: Historical healthcare utilization alone accounts for 96.6% of the total discriminatory power of Logistic Regression.

Running Pipeline Scripts

1. Generating Manuscript Tables & Figures

python generate_manuscript_tables.py
python generate_shap_4panel.py
python generate_shap_blowup_analysis.py
python build_dr_rubin_deliverables.py

2. Running Unit Tests

pytest -v

License & Authorship

Developed by the Lewis Katz School of Medicine at Temple University Center for Biostatistics & Epidemiology in collaboration with the eDERRI Research Group.