DriftSurvival: Survival Analysis on Credit Risk with Data Drift Handling

driftsurvival is a specialized Python package designed for credit risk survival analysis under non-stationary environments. It implements the LMISO (Landmark One-Hot + Isotonic Calibration) framework for mortgage default prediction, addressing data drift caused by macroeconomic shifts, changing borrower behaviors, and dynamic loan lifecycles.

🌟 Key Features

• Longitudinal Behavioral Markers (driftsurvival.markers): Computes balance deviation ($BD_{pct}$) trajectories based on standard loan amortization formulas, tracking borrower payment behavior over time.
• Landmark Dataset Construction (driftsurvival.landmark): Transforms dynamic panel loan data into landmark-based observation points with customizable prediction horizons (e.g., default within 12 months).
• Discrete-Time Hazard Modeling (driftsurvival.models): Logistic regression-based survival models incorporating static, dynamic, and landmark one-hot ($LM$) baseline hazard adjustments.
• Isotonic Probability Calibration (driftsurvival.calibration): Non-parametric post-hoc calibration ensuring accurate default risk probabilities without altering rank ordering.
• Drift Adaptation & Weighting (driftsurvival.models.weighting): Supports Time-Decay Weighting ($TD$) and Importance Weighting ($IW$) for adaptive learning during distribution shifts.
• Drift Simulation & Diagnostics (driftsurvival.drift): Tools for injecting sudden, incremental, or recurring synthetic drift into panel data, along with drift quantification metrics.
• Benchmark Suite (driftsurvival.benchmarks): Standardized wrappers for baseline survival models, including Cox Proportional Hazards (lifelines), XGBoost, and River streaming estimators.
• Visualization Suite (driftsurvival.visualization): Integrated diagnostic plots for survival curves, reliability diagrams, drift distributions, and model coefficients.

🏗️ Package Architecture

driftsurvival/
├── markers/           # Longitudinal marker calculation (BD_pct & trajectory fitting)
├── landmark/          # Dynamic landmark dataset construction & one-hot encoding
├── models/            # Discrete-time hazard models, LMISO estimator, and decay/importance weighting
├── calibration/       # Post-hoc isotonic calibration mapping
├── drift/             # Synthetic drift simulation (sudden, incremental, recurring) & quantification
├── evaluation/        # Survival metrics (AUC, Brier score, ECE) & grouped loan-level CV
├── preprocessing/     # Domain-specific mortgage data preprocessors & amortization calculators
├── visualization/     # Diagnostic plotting utilities
└── benchmarks/        # Interfaces for baseline comparisons (Cox, XGBoost, River)

💻 Installation

Standard Installation

pip install driftsurvival

Installation with Optional Benchmarks & Development Dependencies

# Install with benchmark comparison dependencies (lifelines, xgboost, river)
pip install driftsurvival[benchmarks]

# Install in editable mode for development
pip install -e .[dev,benchmarks]

🚀 Quickstart & Usage

1. Compute Longitudinal Behavioral Markers

import pandas as pd
from driftsurvival.markers import BalanceDeviationMarker

# Sample loan panel data
panel_df = pd.DataFrame({
    "loan_id": ["L001"] * 6,
    "LoanAge": [1, 2, 3, 4, 5, 6],
    "CurAct_UPB": [99500, 99000, 98400, 97800, 97000, 96000],
    "OrigUPB": [100000] * 6,
    "OrigInterestRate": [6.0] * 6,
    "OrigLoanTerm": [360] * 6
})

marker = BalanceDeviationMarker()
df_with_markers = marker.fit_transform(panel_df)
print(df_with_markers[["loan_id", "LoanAge", "BD_pct", "BD_slope"]])

2. Build Landmark Datasets

from driftsurvival.landmark import LandmarkDatasetConstructor

constructor = LandmarkDatasetConstructor(
    landmark_months=[6, 12, 18, 24],
    prediction_horizon=12
)

# Convert dynamic loan history into landmark observations
landmark_data = constructor.transform(
    df=panel_df,
    id_col="loan_id",
    time_col="LoanAge",
    target_col="DefaultFlag"
)

3. Fit the Complete LMISO Estimator

from driftsurvival.models import LMISOEstimator

estimator = LMISOEstimator(
    landmark_months=[6, 12, 18, 24],
    prediction_horizon=12,
    use_calibration=True,
    l2_regularization=1.0
)

# Fit pipeline on historical loan panel
estimator.fit(panel_df, target_col="DefaultFlag", id_col="loan_id", time_col="LoanAge")

# Predict probability of default within horizon at landmark 12
probs = estimator.predict_proba(panel_df, landmark_month=12)

4. Inject Synthetic Data Drift for Testing

from driftsurvival.drift import DriftSimulator

simulator = DriftSimulator(seed=42)

# Inject incremental interest rate & default prevalence drift starting at month 24
drifted_panel = simulator.inject_incremental_drift(
    panel_df,
    feature="OrigInterestRate",
    start_time=24,
    end_time=48,
    magnitude=2.5
)

📊 Evaluation & Metrics

driftsurvival provides group-aware evaluation tools ensuring loan-level isolation during validation:

from driftsurvival.evaluation import GroupedSurvivalCV, evaluate_survival_predictions

metrics = evaluate_survival_predictions(
    y_true=y_test,
    y_prob=probs,
    landmarks=landmark_test_ids
)

print(f"AUC-ROC: {metrics['auc']:.4f}")
print(f"Brier Score: {metrics['brier_score']:.4f}")
print(f"Expected Calibration Error (ECE): {metrics['ece']:.4f}")

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

📖 Citation

This package is based on the following paper:

Incorporating data drift to perform survival analysis on credit risk Jianwei Peng (1), Stefan Lessmann (1 and 2) ((1) Humboldt-Universität zu Berlin, (2) Bucharest University of Economic Studies)

Survival analysis has become a standard approach for modelling time to default by time-varying covariates in credit risk. Unlike most existing methods that implicitly assume a stationary data-generating process, in practise, mortgage portfolios are exposed to various forms of data drift caused by changing borrower behaviour, macroeconomic conditions, policy regimes and so on. This study investigates the impact of data drift on survival-based credit risk models and proposes a dynamic joint modelling framework to improve robustness under non-stationary environments. The proposed model integrates a longitudinal behavioural marker derived from balance dynamics with a discrete-time hazard formulation, combined with landmark one-hot encoding and isotonic calibration. Three types of data drift (sudden, incremental and recurring) are simulated and analysed on mortgage loan datasets from Freddie Mac. Experiments and corresponding evidence show that the proposed landmark-based joint model consistently outperforms classical survival models, tree-based drift-adaptive learners and gradient boosting methods in terms of discrimination and calibration across all drift scenarios, which confirms the superiority of our model design.