zugzwang 1.1.0

Certified selective classification on top of any classifier. Your model shouldn't be forced to move.

zugzwang

Certified selective classification on top of any classifier. Your model shouldn't be forced to move.

In chess, zugzwang is the obligation to move when every move worsens your position. A standard classifier lives in permanent zugzwang: forced to predict even when every prediction is bad. zugzwang gives your model the move the rules deny it — abstention — with statistical risk guarantees. It does not train models: it wraps the one you already have.

from zugzwang import SelectiveClassifier

clf = SelectiveClassifier(LGBMClassifier())              # any predict_proba
clf.fit(X_train, y_train)                                # standard sklearn fit
clf.certify(X_select, y_select, risk=0.02, delta=0.05)   # risk <= 2%, conf. 95%
pred = clf.predict(X)                                    # labels or ABSTAIN (-1)

No wrapper needed (neural nets, LLM logprobs, legacy systems):

from zugzwang.core import certify_threshold

thr = certify_threshold(scores_select, errors_select, risk=0.02, delta=0.05,
                        method="sgr")                    # or "crc"
accept = scores_test >= thr.threshold

The falsifiable contract — every verdict published, win or lose

The product was benchmarked against a contract frozen before the first run: 15 datasets (12 from the preceding research trilogy + 3 external OpenML-CC18), 5 seeds, r* ∈ {0.01, 0.02, 0.05, 0.10}, paired Wilcoxon p<0.05 pre-declared. Full operationalization, amendment trail and losing verdicts included: benchmarks/reports/.

Criterion	Verdict	Key numbers
B1 validity (blocking)	PASS	sgr 9/217 violations (16 allowed); crc 15/294; conformal 21/295
B2 floor vs artisanal	GRAY	Pareto-undominated: +13.7 pp coverage vs the risk-respecting recipe (p=8e-34); the coverage-matching recipe violates risk 38% of the time
B3 ceiling vs MAPIE	REFUTED	MAPIE's sets certify more median coverage; we document when to use MAPIE
B4 budget refund	REFUTED	claim retired; diagnostic: the calibration tax is ~nil — select-split size is what matters

An exploratory follow-up on non-frozen data (X1) decomposed the B3 gap: the method gap vs MAPIE is exactly zero (identical LAC machinery, 300 paired runs) — the whole gap is the price of the high-probability flavor. What that price buys: at the same r*, marginal set readings exceeded the target selective risk on 62% of eval runs; sgr on 3.8%. Score choice and the hybrid budget split moved certified coverage ≤ 0.2 pp: the defaults stand.

With and without, in one picture

XGBoost on satimage (5 seeds, exploratory protocol — the frozen test stays untouched): alone, the model is forced to answer everything at ~9–10% error (black stars). Wrapped, it slides down the risk–coverage frontier and each zugzwang point sits at its certified target (dashed line of its color). The MAPIE selective reading answers a bit more but lands where the marginal guarantee lands it — sometimes past the target. Reproduce: python benchmarks/fig_demo.py satimage.

Three risk notions, one promise

All defaults carry the same high-probability flavor — with confidence ≥ 1−δ over the select sampling, the certified quantity is controlled — and spend at most half the stated error budget (structural safety margin: the stated confidence is a floor, never an estimate).

Method	Certified quantity (confidence ≥ 1−δ)	Machinery
sgr (default)	P(error | accepted) ≤ r*	Exact binomial bound, hybrid search (Geifman & El-Yaniv 2017)
crc	P(error AND accepted) ≤ r*	RCPS (Bates et al. 2021)
conformal	set coverage ≥ 1−r*	PAC / training-conditional (Vovk 2012; Park et al. 2020)

In-expectation / marginal variants (classic CRC, classic split conformal) remain available as explicit opt-in via delta=None, labeled verbatim in the certificate.

The honest data protocol

Train with your full budget; reserve a small labeled select split only to certify. Our own benchmark refuted the stronger version of this claim (B4): what actually matters is the select-split size — certifying power scales with it. UncertifiableRiskError.min_certifiable_risk tells you the floor your data can back, and min_select_size(risk, delta, p_err) plans the budget before you reserve anything:

from zugzwang import min_select_size

min_select_size(risk=0.02, delta=0.05)              # 183 — absolute floor
min_select_size(risk=0.02, delta=0.05, p_err=0.01)  # 851 — realistic budget

Loud, early errors

• Unreachable risk → UncertifiableRiskError reporting the minimum certifiable risk for your data. Never silent thresholds.
• Select rows duplicating training rows → contamination error before any certificate is issued.

What zugzwang is NOT

• It does not train engines (LightGBM/XGBoost are optional extras, never dependencies).
• It does not learn uncertainty scores — nothing learned on top of the model's own confidence paid, measured three ways in the trilogy behind this product. Bring a custom score as a callable if you want.
• It does not detect distribution drift. The production monitor (v1.0) watches observed coverage/risk against the certificate — measurable quantities — and alarms on deviation, with no epistemic claims.

Install

pip install zugzwang          # coming with v0.1.0

Core dependencies: numpy, scipy, scikit-learn. Docs: quickstart, tutorials (sklearn / LightGBM / raw scores), theory per guarantee, and the benchmark pages — see docs/ (GitHub Pages once public).

Phases

Phase	Deliverable	Release	Status
F0	Packaging, CI, frozen benchmark data	—	✅
F1	Certified core + SelectiveClassifier + contract B1–B4 run and published	v0.1 → PyPI	✅
F2	Opt-in calibration + self-contained HTML report	v0.2	✅
F3	DecisionGate + monitoring	v1.0	✅

The companion book (Spanish): book/libro.pdf.

License

MIT