benchleak 0.4.0

Detect benchmark contamination in large language models

benchleak 🔍

Did this model train on the test set? Find out in one command.

When a model scores 90% on GSM8K or MATH, was it genuinely capable or did it memorise the benchmark during training? Benchleak answers that with a mathematical membership-inference test on the model's own token probabilities. No LLM judges, no API calls, runs locally on any HuggingFace causal LM.

Status

🚧 Early alpha. All three detectors are implemented and runnable end to end: the pre-training detector (Min-K% Prob), the fine-tuning detector (probabilistic variation, SPV-MIA), and the RL-post-training detector (Self-Critique entropy).

Install

pip install benchleak

This pulls in torch, transformers, and datasets. To work from a clone instead, run pip install -e . in the repo root.

Usage

benchleak --model Qwen/Qwen2.5-0.5B --benchmark gsm8k

The model can be any HuggingFace-format causal LM, given as a Hub id or a local checkpoint directory. GGUF, llama.cpp, and Ollama formats are not supported.

Private or gated models

For a repository that requires authentication, provide a HuggingFace token. Any of these work:

huggingface-cli login                  # cached credential, picked up automatically
export HF_TOKEN=hf_xxx                  # environment variable
benchleak --model my/private-model --benchmark gsm8k --hf-token hf_xxx

Speed

Scoring runs one forward pass per sample. On a CPU-only machine the default --limit 200 can take many minutes; use a smaller --limit for a quick look, or --device cuda / --device mps to use a GPU.

benchleak: contamination report
====================================================
Model:       Qwen/Qwen2.5-0.5B
Benchmark:   gsm8k
Detector:    min-k% prob
Samples:     40 benchmark vs 40 reference
Reference:   bundled reference-math.txt

Separation (AUC):   0.580   [LOW]
Significance (p):   0.11
Flag thresholds:    AUC >= 0.6, p < 0.05

Verdict: NO STRONG EVIDENCE

The benchmark is compared against a reference set matched to its domain and format — for GSM8K, original math word problems written for this project. That matters: against a general-prose reference the same scan reports AUC 0.98 and flags the model, but that separation is almost entirely math vs prose, not seen vs unseen. A domain-matched reference isolates the memorisation signal.

Known benchmarks (gsm8k, math, arc-challenge, truthfulqa) work by name. For any other Hub dataset, pass the path plus its text column(s):

benchleak --model my/model --benchmark some/dataset --field question --field answer

Your own benchmark from a local file

Point --benchmark at a local file instead of a Hub id. Supported formats:

• .txt: one passage per line (no --field needed)
• .jsonl / .json / .csv: name the text column(s) with --field

benchleak --model my/model --benchmark ./my_benchmark.jsonl --field question --field answer

Local files are read with the standard library, so this path needs neither a network connection nor the datasets package.

Choosing a detector

--detector pretrain (the default) runs Min-K% Prob, which targets memorisation from pre-training. --detector sft runs probabilistic variation (SPV-MIA), which targets memorisation from fine-tuning and is the right choice when you suspect a model was fine-tuned on a benchmark:

benchleak --model my/model --benchmark gsm8k --detector sft

The fine-tuning detector paraphrases each sample and compares log-likelihoods, so it is roughly an order of magnitude slower than the pre-training detector. By default it downloads a T5 paraphrasing model; pass --perturber word to avoid the download (lower quality) and --n-perturbations to trade speed for stability. See docs/how-it-works-sft.md for the method and caveats.

--detector rl runs Self-Critique, which targets memorisation from RL post-training (RLVR/GRPO) — the phase where likelihood-based detectors fail. It is the right choice for an RL/reasoning-tuned instruct model:

benchleak --model my/model --benchmark gsm8k --detector rl

Here each sample is treated as a problem to solve: the detector generates a response, asks the model to redo it along a different reasoning path, and measures how similar the two answers' entropy curves stay (a contaminated problem can't deviate). Because it generates two responses per sample it is the slowest detector — use a small --limit and tune --max-new-tokens. It needs a model with a chat template. See docs/how-it-works-rl.md.

How it works

The benchmark is scored against a reference set of text the model is not expected to have memorised. Min-K% Prob assigns each text the mean log-probability of its least-likely k% of tokens. Memorised text has fewer surprising tokens and scores higher. The tool then measures how strongly the benchmark's scores separate from the reference's, reported as an AUC (U / nm from a Mann-Whitney test) with a significance p-value. AUC ≈ 0.5 means the benchmark looks like fresh data; AUC well above 0.5 is the memorisation signature of contamination.

Bundled reference sets ship with the tool so it runs out of the box, and the right one is picked by the benchmark's domain: math benchmarks (gsm8k, math) are compared against original GSM8K-style word problems written for this project (never published, so no model trained on them — and their arithmetic is machine-verified by the test suite); everything else uses a general-prose set. For the cleanest signal, supply your own domain-matched reference data with --reference my_reference.txt (one passage per line).

For the full reasoning (why a reference set is needed, the choice of test, and where the method can mislead), see docs/how-it-works-pretrain.md.

Phase	Method	Status
Pre-training	Min-K% probability (Shi et al. 2024)	✅ implemented
SFT	Probabilistic variation / SPV-MIA (Fu et al. 2024)	✅ implemented
RL post-training	Self-Critique entropy (Tao et al. 2025)	✅ implemented

Caveats

• A verdict needs ≥ 5 samples per side; the significance test cannot reach p < 0.05 below that.
• Math benchmarks are automatically compared against the bundled math reference. For any other narrow-domain benchmark the fallback is general-domain prose, which can confound domain with memorisation; prefer a domain-matched --reference for results you intend to publish.

Troubleshooting

ModuleNotFoundError: No module named '_lzma' when loading a benchmark. The datasets library needs Python's lzma module, which is absent from some Python builds (commonly pyenv on macOS compiled without the xz library). Install xz and rebuild Python:

brew install xz
LDFLAGS="-L$(brew --prefix xz)/lib" CPPFLAGS="-I$(brew --prefix xz)/include" \
  pyenv install -f <your-python-version>

Then reuse or recreate your virtual environment. benchleak detects this case and prints the same guidance.

Citation

Implements methods from:

• Shi et al., 2024. Detecting Pretraining Data from Large Language Models (arXiv:2310.16789)
• Fu et al., 2024. Membership Inference via Self-Prompt Calibration
• Tao et al., 2025. Detecting Data Contamination from RL Post-training (arXiv:2510.09259)

License

Apache-2.0