purgedcv 0.0.10

scikit-learn-compatible cross-validation for time-series and financial machine learning: purging, embargoes, combinatorial purged CV, and deflated Sharpe ratios.

Purged cross validation

scikit-learn-compatible cross-validation for time-series machine learning: purging, embargoes, and combinatorial backtest paths.

Documentation → · Example notebooks → — purge/embargo, walk-forward, and CPCV with PSR/DSR worked end to end on real ICU-mortality, turbofan-RUL, rainfall, and electricity-demand data.

Cite this software: see CITATION.cff and paper/paper.md (JOSS paper).

---

The problem

Standard k-fold cross-validation assumes the rows are independent. Time-series data is not. When a label resolves over the next few days, it overlaps the labels sitting right next to it, so an ordinary shuffle-split leaks tomorrow's answer back into training. The rows immediately after a test window leak too, because they are serially correlated with it. Both effects quietly inflate backtested Sharpe ratios and hand you strategies that look great on a chart and bleed money once they go live. This library removes both.

Why write another one? People have asked scikit-learn, auto-sklearn, and mlpack for purging and embargo support and been turned down or left waiting for years. The one mature implementation, mlfinlab, went closed-source and paid. The free alternative has been unmaintained since 2018. That gap is the reason this exists.

---

Does it actually catch leakage?

A controlled check on synthetic data whose target is built so that no feature can predict it. The honest out-of-sample score must never be positive. Naive shuffled k-fold runs against PurgedKFold side by side (examples/synthetic_leakage_proof.ipynb, deterministic, no download):

model	naive shuffled KFold R²	PurgedKFold R²
predict-the-mean (reference)	-0.01	-0.13
k-NN	0.83	-1.31
RandomForest	0.91	-1.94

Train/test label overlap: 100% under naive → 0% under PurgedKFold.

Naive CV reports R² ≈ 0.83–0.91 on a target nothing can predict. That is pure leakage from the overlap. PurgedKFold removes the overlap and the fabricated skill collapses below a predict-the-mean baseline. The negative number is not the point; no positive skill is the correct answer, and only the purged split reports it. The library does not make models look better; it stops them looking better than they are.

---

Installation

pip install purgedcv

# Directly from the repository
pip install git+https://github.com/eslazarev/purged-cross-validation.git

---

Quickstart

1. The core primitive: purge

purge removes training observations that share data with the test set. Here a model uses a 5-day sliding feature window to predict the next day, so every observation occupies a 5-day span and the spans of neighbours overlap. Any training observation whose window reaches into the test period has already seen test data and must be dropped.

import numpy as np
import pandas as pd
from purgedcv import purge

WINDOW = 5  # feature look-back in days

# 16 days of data; each observation uses a 5-day window to predict the next day
days        = pd.date_range("2024-01-01", periods=16, freq="D")
predict_day = np.arange(WINDOW + 1, len(days) + 1)                    # 11 observations
pred        = pd.Series([days[d - WINDOW - 1] for d in predict_day])  # first feature day
evalu       = pd.Series([days[d - 1] for d in predict_day])           # label day

train_idx = np.arange(0, 7)    # observations predicting days 6..12
test_idx  = np.arange(7, 11)   # observations predicting days 13..16

# Drop training observations whose 5-day feature window overlaps the test window
kept_idx   = purge(train_idx, test_idx, pred, evalu)
purged_idx = np.setdiff1d(train_idx, kept_idx)

print(f"Kept:   {kept_idx.tolist()}")    # [0, 1, 2]    -> predict days 6, 7, 8
print(f"Purged: {purged_idx.tolist()}")  # [3, 4, 5, 6] -> predict days 9, 10, 11, 12

Each bar below is one observation's 5-day feature window. The four red bars cross into the test window (dashed line) — their features overlap the test period, so purge drops them. The three green bars stay fully before it; → day 8 only touches the boundary and is kept, because label horizons are half-open.

---

2. The post-test buffer: apply_embargo

apply_embargo is the second leakage guard. Where purge removes label overlap, embargo drops a fixed buffer of training rows right after the test fold — their features are still serially correlated with the test period, so they leak even when their labels do not. The buffer extends only after the test, never before it.

import numpy as np
import pandas as pd
from purgedcv import apply_embargo

# 16 daily rows; each label resolves 1 day later
n     = 16
pred  = pd.Series(pd.date_range("2024-01-01", periods=n, freq="D"))
evalu = pred + pd.Timedelta(days=1)

test_idx  = np.arange(6, 10)                                      # test block in the middle
train_idx = np.concatenate([np.arange(0, 6), np.arange(10, 16)])  # train before and after

# Drop training rows whose prediction time falls in the post-test buffer
kept_idx      = apply_embargo(train_idx, test_idx, pred, evalu, embargo=pd.Timedelta(days=3))
embargoed_idx = np.setdiff1d(train_idx, kept_idx)

print(f"Kept:      {kept_idx.tolist()}")       # [0, 1, 2, 3, 4, 5, 14, 15]
print(f"Embargoed: {embargoed_idx.tolist()}")  # [10, 11, 12, 13]

Each bar below is one row's label horizon. Every training row whose start falls inside the orange +3d buffer after the test is dropped; rows before the test are untouched. The 1-day label horizon leaves purge nothing to do here, so the split is cleaned by embargo alone.

---

3. Walk-forward CV: WalkForwardSplit

WalkForwardSplit walks the train/test split forward in time: every fold trains only on data before its test block, the way a model is actually deployed. Purge and embargo are applied automatically on each fold. Use window="expanding" to keep all history, or window="sliding" to cap training at a fixed-size recent window.

import numpy as np
import pandas as pd
from purgedcv import WalkForwardSplit

# 24 daily rows; each label resolves 2 days later
n     = 24
pred  = pd.Series(pd.date_range("2024-01-01", periods=n, freq="D"))
evalu = pred + pd.Timedelta(days=2)
X     = np.zeros((n, 1))

# Expanding window: every fold trains on all data before its test block.
# purge_horizon drops the training rows whose label overlaps each test fold.
cv = WalkForwardSplit(
    n_splits=3,
    test_size=4,
    window="expanding",          # or "sliding" with train_size=...
    prediction_times=pred,
    evaluation_times=evalu,
    purge_horizon="2D",
)

for i, (train_idx, test_idx) in enumerate(cv.split(X), 1):
    print(f"fold {i}: train {train_idx[0]}..{train_idx[-1]}  test {test_idx[0]}..{test_idx[-1]}")
# fold 1: train 0..8   test 12..15
# fold 2: train 0..12  test 16..19
# fold 3: train 0..16  test 20..23

Three folds tile the end of the series. Each fold trains on the past and tests on the next block; the red purge gap right before each test is removed automatically. Expanding grows the training set every fold; sliding keeps it a fixed size and moves it forward.

---

4. Purged k-fold: PurgedKFold

A drop-in replacement for KFold for time-series data. Test folds tile the whole series contiguously, so most folds sit inside the data and the training set wraps around them on both sides. Purge and embargo are applied automatically on every fold.

import numpy as np
import pandas as pd
from purgedcv import PurgedKFold

# 24 daily rows; each label resolves 2 days later
n     = 24
pred  = pd.Series(pd.date_range("2024-01-01", periods=n, freq="D"))
evalu = pred + pd.Timedelta(days=2)
X     = np.zeros((n, 1))

# 4 contiguous test folds; purge + embargo applied automatically per fold
cv = PurgedKFold(
    n_splits=4,
    prediction_times=pred,
    evaluation_times=evalu,
    purge_horizon="2D",
    embargo="2D",
)

for i, (train_idx, test_idx) in enumerate(cv.split(X), 1):
    print(f"fold {i}: test {test_idx[0]}..{test_idx[-1]}  train rows {len(train_idx)}")
# fold 1: test 0..5    train rows 14
# fold 2: test 6..11   train rows 11
# fold 3: test 12..17  train rows 11
# fold 4: test 18..23  train rows 15

Each test fold is a contiguous block. A middle fold trains on data both before and after it; the red purge gap before the test and the orange purge + embargo buffer after it are removed automatically, so no training row leaks into the fold.

All four splitters (WalkForwardSplit, PurgedKFold, PurgedGroupKFold, CombinatorialPurgedCV) satisfy the sklearn splitter protocol and work inside cross_val_score, GridSearchCV, and Pipeline.

---

5. CPCV + path reconstruction + metrics: the full workflow

Combinatorial Purged CV produces C(N, K) folds that tile into multiple out-of-sample backtest paths. Use PSR and DSR to evaluate them with corrections for non-normality and selection bias.

import numpy as np
import pandas as pd
from sklearn.dummy import DummyRegressor
from purgedcv import (
    CombinatorialPurgedCV,
    probabilistic_sharpe_ratio,
    deflated_sharpe_ratio,
    min_track_record_length,
)

rng   = np.random.default_rng(42)
n     = 120
pred  = pd.Series(pd.date_range("2023-01-01", periods=n, freq="D"))
evalu = pred + pd.Timedelta(days=2)
X     = rng.standard_normal((n, 3))
y     = X @ np.array([0.5, -0.3, 0.2]) + rng.standard_normal(n) * 0.1

# N=6, K=2  →  C(6,2) = 15 folds  →  C(5,1) = 5 backtest paths
cv = CombinatorialPurgedCV(
    n_splits=6,
    n_test_groups=2,
    prediction_times=pred,
    evaluation_times=evalu,
)

# paths.shape == (n_paths, n_samples); NaN only if a fold could not be fit
paths = cv.backtest_paths(DummyRegressor(strategy="mean"), X, y)
print(f"Backtest paths: {paths.shape}")  # (5, 120)

# Derive a toy "return" series and compute per-path PSR
per_path_returns = paths - y[np.newaxis, :]
per_path_psr = [
    probabilistic_sharpe_ratio(row[np.isfinite(row)], benchmark_skill=0.0)
    for row in per_path_returns
]
print(f"PSR per path: {[round(p, 3) for p in per_path_psr]}")

# DSR corrects for testing 5 paths simultaneously
first = per_path_returns[0]
dsr = deflated_sharpe_ratio(first[np.isfinite(first)], n_trials=5, var_sharpe=0.01**2)
print(f"Deflated SR (first path): {dsr:.3f}")

# Minimum observations needed to prove SR=0.7 beats benchmark SR=0.5 at 95% confidence
n_min = min_track_record_length(
    observed_sharpe=0.7, target_sharpe=0.5, alpha=0.05, skew=0.0, kurtosis=3.0
)
print(f"MinTRL: {int(n_min)} observations")

---

API summary

Symbol	Domain	Description
purge	D2	Remove overlapping-horizon training rows
apply_embargo	D3	Remove post-test buffer rows
WalkForwardSplit	D5.1	Sliding / expanding walk-forward CV
PurgedKFold	D5.2	Contiguous test folds with purge + embargo
PurgedGroupKFold	D5.3	Group-aware purged k-fold
CombinatorialPurgedCV	D5.4	C(N,K) combinatorial folds
reconstruct_paths	D6	Assemble CPCV folds into backtest paths
probabilistic_sharpe_ratio	D7	PSR: P(true SR > benchmark)
deflated_sharpe_ratio	D7	DSR: PSR corrected for multiple testing
min_track_record_length	D7	Minimum observations to establish SR
diagnostics.*	D8	Leakage and embargo audit functions

---

Methodology references

• Lopez de Prado, M. (2018). Advances in Financial Machine Learning. Wiley. Chapters 7 (purge/embargo) and 12 (CPCV).
• Bailey, D. H., & Lopez de Prado, M. (2012). The Sharpe Ratio Efficient Frontier. Journal of Risk, 15(2).
• Bailey, D. H., & Lopez de Prado, M. (2014). The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality. Journal of Portfolio Management, 40(5).

---

License

MIT. See LICENSE.