foldguard

Your train/test split is fine. Your numbers are still too good.

There's a family of leaks that never shows up in a diff of the modelling code, because the split itself is textbook. The eval fold never touches training. It just quietly steers decisions:

• you kept the epoch that scored best on the test fold (FG-1)
• you fit the calibrator on the labels you then report against (FG-2)
• you picked the threshold by maximising F1 on the scores you're about to publish (FG-3)

Each of those inflates the number you report, and none of them look wrong when you read the code. Existing leakage tools go after preprocessing and feature leakage. Nothing I could find catches these at runtime, so I wrote this.

Version 0.1.0, the first release. Early enough that the API may still move.

Getting started

pip install foldguard[sklearn]

You mark the eval fold once (the library calls it tainting, after the taint-tracking idea it borrows from security tooling) and run your protocol inside a guard. The snippet below works as-is:

import foldguard as fg
from sklearn import metrics as skm
from sklearn.datasets import make_classification
from sklearn.isotonic import IsotonicRegression
from sklearn.linear_model import LogisticRegression

X, y = make_classification(n_samples=400, n_features=20, random_state=0)
X_tr, X_te, y_tr, y_te = fg.sklearn.tainted_train_test_split(
    X, y, test_size=0.2, random_state=0
)

with fg.guard(action="record") as g:      # "raise" (the default) stops at the first one
    model = LogisticRegression(max_iter=1000).fit(X_tr, y_tr)
    scores = model.predict_proba(X_te)[:, 1]
    auroc = fg.report("auroc", skm.roc_auc_score(y_te, scores))

    IsotonicRegression().fit(scores, y_te)         # FIT_ON_EVAL (FG-2)
    if skm.roc_auc_score(y_te, scores) > 0.5:      # DECISION_ON_EVAL (FG-1)
        best_model = model

print(g.summary())

The first three lines inside the guard are fine. Fitting on the training half, scoring the eval fold, reporting the result: that's what a test set is for. The last two are not, and you get the file and line for each, plus a note on what the honest version looks like.

One rule covers it. Computing on eval data is fine. Letting eval data change what your pipeline does is not.

One wrinkle worth knowing early: use from sklearn import metrics as skm, not from sklearn.metrics import roc_auc_score. A from-import that runs before any guard exists grabs the unwrapped function, so nothing downstream of it carries a mark. The foldguard run CLI and the pytest plugin patch things before your imports and don't have this problem. In a plain script you'll get a FoldguardEarlyBindWarning if you trip it, so at least it's loud.

What it watches

numpy. Marks propagate through ufuncs, slicing and dispatched np.* calls. Two sinks: bool() of a marked value (DECISION_ON_EVAL) and the argmax/argsort family (SELECT_ON_EVAL). Per-sample class prediction (probs.argmax(axis=1)) is exempt, since that's a prediction and not a choice. Flattening argmax, which is how threshold tuning looks, still fires.

sklearn. Every BaseEstimator fit method is a sink. predict, transform, score and all of sklearn.metrics carry marks through instead of complaining. Data arriving through eval_set-style keywords gets its own message, because that's early stopping on the test fold rather than fitting on it, and the fix is different.

xgboost and lightgbm, including the native APIs. xgb.train/DMatrix and lgb.train/Dataset are covered, and the check happens before any boosting round. The field test is why: the sklearn wrappers were caught from day one while the native path sailed straight through.

pandas. fg.taint() takes a Series or DataFrame and hands one back. The mark survives slicing, masking, arithmetic and train_test_split, and fit guards see it. .to_numpy() still drops it, which is in the limitations below.

Audit output is deduplicated by location with counts, so a loop that trips the same line twenty thousand times costs you three lines of summary rather than a megabyte. I know that number because a pipeline in the field test did exactly that.

What a violation actually means

A violation says eval data reached a fit, a branch or a selection on that line. That part is a fact about your run, not a guess.

Whether it's a leak is a separate question, and it's one you answer by looking at the line. Picking a checkpoint that way is a leak. Asserting your class balance is sane before you score is not, even though both branch on eval data. So there are two ways to say "I meant that":

# a deliberate check
with fg.allow("data validation"):
    if np.isnan(y_test).any():
        raise SystemExit("bad labels in the eval fold")

# a hand-rolled metric that has to sort or bin its own inputs
@fg.allow_metric
def ece(y_true, probs, n_bins=10):
    ...

ece_val = ece(y_test, scores)   # still marked, so tuning on it is still caught
auroc = fg.report("auroc", skm.roc_auc_score(y_test, scores))
assert auroc > 0.7              # report first, then assert on the plain float

Both get logged with a reason and a location, so anyone auditing the run can see what you waved through and decide whether they agree.

The reverse doesn't hold, and you should hold this against the tool: a clean run is not proof of a clean protocol. See the limitations.

The three leaks, side by side

$ python examples/hero.py
foldguard: three textbook evaluation leaks, one guard
=====================================================

leak   protocol             metric    leaky  honest  optimism   foldguard verdict
-------------------------------------------------------------------------------------------
FG-1   checkpoint on test   AUROC     0.789   0.726    +0.063   LeakError: DECISION_ON_EVAL
FG-2   calibrator on test   AUROC     0.767   0.734    +0.033   LeakError: FIT_ON_EVAL
FG-3   threshold on test    F1        0.727   0.681    +0.046   LeakError: SELECT_ON_EVAL
-------------------------------------------------------------------------------------------
Same data, same model, same split in every row. The only difference is
whether the test fold was allowed to steer a choice.

Don't read too much into the size of those gaps. They're one pinned seed. Selection optimism is never negative but it can be zero if you get lucky, and on FG-1 four of ten held-out seeds came out at exactly +0.000, because validation happened to pick the same epoch the test fold would have. The protocol is broken either way. Some seeds just don't send you the bill.

In CI

@pytest.mark.leakfree
def test_eval_protocol():
    run_full_evaluation()

foldguard run eval_pipeline.py --report foldguard-report.json

foldguard run exits 0 when clean, 2 on violations, 1 if your script raised, 64 for bad usage. A script that opens its own guard runs fine under it; the inner guard folds into the outer audit instead of erroring. There's a GitHub Action in action.yml.

Does it work on real code?

I ran it against 30 public evaluation pipelines across about 20 domains before releasing it: 10 executed directly, 20 re-typed faithfully from their eval code. 23 had genuine protocol leaks. The other 7 were pipelines I picked because they looked clean, to find out how often it cries wolf. The corpus stays anonymous, since the point was to test my tool rather than to publish an audit of other people's work.

It flagged the leak on the exact line in 15 of the 23, and in 20 of 23 once the same leak was expressed on surfaces it can see. The three it can't reach are structural and described below. Median cost to integrate was 6 changed lines, worst case 14.

On the clean pipelines it stayed quiet about protocol, which is the result I cared about most: fit-on-validation, nested CV, stacking and internal model selection all passed without a word. It did fire 10 times, every one of them inside a hand-rolled metric helper that had to branch or sort on its own inputs. The most common shape (per-sample argmax) is now exempt, and the rest take a one-line @fg.allow_metric.

The useful finding was that every miss was a transport failure rather than a judgement failure. The sink logic was right every time; the mark just didn't survive the trip, through pandas containers, np.array copies, a framework's internal float(), or a native GBDT entry point. Native GBDT and pandas ingestion are fixed as of this release. The rest are documented route by route, because a leak that audits clean and silent is the worst thing this tool could do.

Limitations

It's a bug finder, not a proof. Two kinds of gap.

Structural. Mark the eval fold before anything supervised touches those rows. If your vectorizer or feature filter is fit on the whole dataset before the split, the leak happens before the mark exists and nothing here can see it. Cross-fold row identity (duplicate, grouped or overlapping rows, FG-7) is a different mechanism and out of scope. And a threshold you picked by eyeballing a plot is invisible to any runtime tool.

Erasure. Some operations drop the mark. Where a hook exists, you get a deduplicated taint-erased event so at least there's a breadcrumb: float(), int(), .item(), .tolist(), formatting or repr() of a marked scalar, np.array([...]) over a list of marked scalars, pandas ingesting marked scalars, and fg.report() handed something already unmarked.

Some are simply invisible, and each one is pinned by a test so it can't quietly change: np.asarray over a marked array (numpy short-circuits subclasses in C), stacking a marked array with a plain one, indexing a plain array with a marked index or mask, writes into plain buffers via out=, np.copyto or slice assignment, Series.to_numpy(), pickling, a framework's own float() boundaries such as HPO trial storage, and anything training outside sklearn.base.BaseEstimator, which means torch and keras loops.

High precision on what it flags. Incomplete recall on what it can reach.

Docs

• Design and architecture
• The leak taxonomy, FG-1 to FG-7
• Integration recipes for pandas, GBDTs, CV loops, HPO without a holdout, and torch

Built with Claude

Claude (Anthropic) did most of the typing on this, via Claude Code. The taxonomy and the design came out of working through the problem with it, and two of the things I'm most confident about were run as multi-agent Claude workflows: an adversarial review that found 21 defects in my first version, including a crash and a sink that could be bypassed entirely, and the 30-pipeline field test above.

I set the scope, made the design calls, and decided what to believe. Every number in this README came from a run I checked, and the claims got weaker rather than stronger each time something was verified properly. The original README said every violation was a true positive. That was wrong, and the review proved it with a counterexample in about a minute.

License

MIT