driftdep 0.1.0

Dependence-robust two-sample drift tests for serially correlated feature streams

driftdep

Your drift monitor is lying to you. If your feature windows are autocorrelated — and they almost always are — standard two-sample tests (KS, CvM, AD, MMD) over-reject by 2–10× at realistic dependencies. A single AR(1) coefficient of ρ=0.7 inflates the KS false-alarm rate from 5% to 35%. Long-memory processes (ARFIMA) push it even higher.

driftdep is a drop-in, dependence-robust replacement that corrects for serial dependence via ESS-adjusted block permutation by default.

# Before — unreliable under serial dependence
from scipy.stats import ks_2samp
stat, p = ks_2samp(x_ref, x_new)   # p-value too small when data is autocorrelated

# After — calibrated by default
from driftdep import ks_2samp
stat, p = ks_2samp(x_ref, x_new)   # same call, same unpacking, correct p-value

Install

pip install driftdep

Requires Python ≥ 3.10, numpy ≥ 1.24, scipy ≥ 1.10 only. Heavy research dependencies (arch, statsmodels, dcor) are optional:

pip install "driftdep[research]"   # for all corrections and download scripts

Quick start

import numpy as np
import driftdep

rng = np.random.default_rng(0)
x_ref = rng.standard_normal(500)
x_new = rng.standard_normal(500)   # same distribution; should not alarm

stat, p = driftdep.ks_2samp(x_ref, x_new)
# p ≈ 0.6 — no false alarm (correct even if the series is autocorrelated)

# Full result with diagnostics
result = driftdep.drift_test(x_ref, x_new, statistic="ks")
print(result.pvalue)        # dependence-corrected p-value
print(result.n_eff)         # effective sample size estimate
print(result.block_length)  # block length used by block permutation

API

driftdep.drift_test(
    x, y, *,
    statistic="ks",           # "ks", "cvm", "ad", "energy", "mmd",
                              # "wasserstein", "psi", "js"
    correction="block_perm",  # default engine; see table below
    block_length=None,        # None → auto (Politis–White 2004)
    ess_adjust=True,          # compute and report indicator-transform ESS
    n_resamples=999,          # permutation resamples
    alpha=0.05,
    rng=None,                 # numpy Generator, int seed, or None
) -> DriftResult

DriftResult unpacks as (statistic, pvalue) for drop-in compatibility and also exposes .pvalue, .n_eff, .block_length, .correction.

Available corrections

correction=	Description	Extra deps
block_perm	Block permutation, Politis–White block length	—
ess_adjust	ESS-adjusted analytic null (cheapest; KS/CvM/AD)	—
naive	i.i.d. null — baseline / reference only	—
thinning	Keep every k-th obs to approximate independence	—
dep_wild	Dependent wild bootstrap (Shao 2010)	—
mbb	Moving block bootstrap (Künsch 1989)	[research]
cbb	Circular block bootstrap	[research]
stationary	Stationary bootstrap (Politis–Romano 1994)	[research]
prewhiten	Cautionary only — changes the hypothesis	[research]

Method

driftdep benchmarks and operationalizes dependence corrections for two-sample distributional drift tests. The core insight: when observations within a monitoring window are autocorrelated, the i.i.d. null distribution of KS, CvM, and related statistics is stochastically dominated, causing severe over-rejection.

The recommended default — ESS-adjusted block permutation — permutes contiguous blocks of length b (preserving within-block dependence) and selects b automatically via Politis–White (2004). Empirical size recovers to ≈ α across AR(1), ARFIMA, and GARCH dependence structures. Size-adjusted power is within 5–7 percentage points of the uncalibrated naive test at moderate effect sizes.

All corrections already exist in the literature; this package operationalizes them in a unified interface with a validated default.

Reproduce paper results

git clone https://github.com/vivekch2018/driftdep
cd driftdep
pip install -e ".[research]"
make data        # download public datasets (requires internet)
make reproduce   # regenerate all figures and tables from seeds

Citation

@misc{chaudhary2025driftdep,
  author  = {Chaudhary, Vivek},
  title   = {Two-Sample Drift Tests Break Under Serial Dependence:
             A Benchmark and Deployable Correction},
  year    = {2025},
  note    = {Preprint},
  url     = {https://github.com/vivekch2018/driftdep},
}

License

MIT