Metadata-Version: 2.4
Name: zugzwang
Version: 1.1.0
Summary: Certified selective classification on top of any classifier. Your model shouldn't be forced to move.
Project-URL: Repository, https://github.com/brunomarentes/Zugzwang
Author: Bruno Marentes
License-Expression: MIT
License-File: LICENSE
Keywords: abstention,conformal prediction,machine learning,risk control,selective classification
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24
Requires-Dist: scikit-learn>=1.3
Requires-Dist: scipy>=1.10
Provides-Extra: all
Requires-Dist: crepes>=0.7; extra == 'all'
Requires-Dist: lightgbm>=4.0; extra == 'all'
Requires-Dist: mapie>=1.0; extra == 'all'
Requires-Dist: matplotlib>=3.8; extra == 'all'
Requires-Dist: mkdocs-material>=9.5; extra == 'all'
Requires-Dist: pandas>=2.0; extra == 'all'
Requires-Dist: pyarrow>=15.0; extra == 'all'
Requires-Dist: pytest>=8.0; extra == 'all'
Requires-Dist: pyyaml>=6.0; extra == 'all'
Requires-Dist: xgboost>=2.0; extra == 'all'
Provides-Extra: bench
Requires-Dist: crepes>=0.7; extra == 'bench'
Requires-Dist: lightgbm>=4.0; extra == 'bench'
Requires-Dist: mapie>=1.0; extra == 'bench'
Requires-Dist: matplotlib>=3.8; extra == 'bench'
Requires-Dist: pandas>=2.0; extra == 'bench'
Requires-Dist: pyarrow>=15.0; extra == 'bench'
Requires-Dist: pytest>=8.0; extra == 'bench'
Requires-Dist: pyyaml>=6.0; extra == 'bench'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Provides-Extra: test
Requires-Dist: lightgbm>=4.0; extra == 'test'
Requires-Dist: pytest>=8.0; extra == 'test'
Requires-Dist: xgboost>=2.0; extra == 'test'
Description-Content-Type: text/markdown

# zugzwang

[![ci](https://github.com/brunomarentes/Zugzwang/actions/workflows/ci.yml/badge.svg)](https://github.com/brunomarentes/Zugzwang/actions/workflows/ci.yml)

**Certified selective classification on top of any classifier.**
*Your model shouldn't be forced to move.*

In chess, *zugzwang* is the obligation to move when every move worsens your
position. A standard classifier lives in permanent zugzwang: forced to predict
even when every prediction is bad. **zugzwang** gives your model the move the
rules deny it — abstention — with statistical risk guarantees. It does not
train models: it wraps the one you already have.

```python
from zugzwang import SelectiveClassifier

clf = SelectiveClassifier(LGBMClassifier())              # any predict_proba
clf.fit(X_train, y_train)                                # standard sklearn fit
clf.certify(X_select, y_select, risk=0.02, delta=0.05)   # risk <= 2%, conf. 95%
pred = clf.predict(X)                                    # labels or ABSTAIN (-1)
```

No wrapper needed (neural nets, LLM logprobs, legacy systems):

```python
from zugzwang.core import certify_threshold

thr = certify_threshold(scores_select, errors_select, risk=0.02, delta=0.05,
                        method="sgr")                    # or "crc"
accept = scores_test >= thr.threshold
```

## The falsifiable contract — every verdict published, win or lose

The product was benchmarked against a contract frozen **before** the first
run: 15 datasets (12 from the preceding research trilogy + 3 external
OpenML-CC18), 5 seeds, r\* ∈ {0.01, 0.02, 0.05, 0.10}, paired Wilcoxon
p<0.05 pre-declared. Full operationalization, amendment trail and losing
verdicts included: [`benchmarks/reports/`](benchmarks/reports/contract.md).

| Criterion | Verdict | Key numbers |
|---|---|---|
| **B1 validity** (blocking) | **PASS** | sgr 9/217 violations (16 allowed); crc 15/294; conformal 21/295 |
| **B2 floor** vs artisanal | **GRAY** | Pareto-undominated: +13.7 pp coverage vs the risk-respecting recipe (p=8e-34); the coverage-matching recipe violates risk 38% of the time |
| **B3 ceiling** vs MAPIE | **REFUTED** | MAPIE's sets certify more median coverage; we document when to use MAPIE |
| **B4 budget refund** | **REFUTED** | claim retired; diagnostic: the calibration tax is ~nil — select-split *size* is what matters |

An exploratory follow-up on non-frozen data
([X1](benchmarks/reports/x1_exploratory.md)) decomposed the B3 gap: the
method gap vs MAPIE is **exactly zero** (identical LAC machinery, 300 paired
runs) — the whole gap is the price of the high-probability flavor. What that
price buys: at the same r\*, marginal set readings exceeded the target
selective risk on 62% of eval runs; sgr on 3.8%. Score choice and the hybrid
budget split moved certified coverage ≤ 0.2 pp: the defaults stand.

## With and without, in one picture

![XGBoost with and without zugzwang on satimage](https://raw.githubusercontent.com/brunomarentes/Zugzwang/main/benchmarks/reports/fig_demo_xgb.png)

XGBoost on satimage (5 seeds, exploratory protocol — the frozen test stays
untouched): alone, the model is forced to answer everything at ~9–10% error
(black stars). Wrapped, it slides down the risk–coverage frontier and each
zugzwang point sits at its **certified** target (dashed line of its color).
The MAPIE selective reading answers a bit more but lands where the marginal
guarantee lands it — sometimes past the target. Reproduce:
`python benchmarks/fig_demo.py satimage`.

## Three risk notions, one promise

All defaults carry the same high-probability flavor — with confidence ≥ 1−δ
over the select sampling, the certified quantity is controlled — and spend at
most **half** the stated error budget (structural safety margin: the stated
confidence is a floor, never an estimate).

| Method | Certified quantity (confidence ≥ 1−δ) | Machinery |
|---|---|---|
| `sgr` (default) | P(error \| accepted) ≤ r\* | Exact binomial bound, hybrid search (Geifman & El-Yaniv 2017) |
| `crc` | P(error AND accepted) ≤ r\* | RCPS (Bates et al. 2021) |
| `conformal` | set coverage ≥ 1−r\* | PAC / training-conditional (Vovk 2012; Park et al. 2020) |

In-expectation / marginal variants (classic CRC, classic split conformal)
remain available as explicit opt-in via `delta=None`, labeled verbatim in
the certificate.

## The honest data protocol

Train with your full budget; reserve a small labeled **select** split only to
certify. Our own benchmark refuted the stronger version of this claim (B4):
what actually matters is the select-split *size* — certifying power scales
with it. `UncertifiableRiskError.min_certifiable_risk` tells you the floor
your data can back, and `min_select_size(risk, delta, p_err)` plans the
budget before you reserve anything:

```python
from zugzwang import min_select_size

min_select_size(risk=0.02, delta=0.05)              # 183 — absolute floor
min_select_size(risk=0.02, delta=0.05, p_err=0.01)  # 851 — realistic budget
```

## Loud, early errors

- Unreachable risk → `UncertifiableRiskError` reporting the minimum
  certifiable risk for your data. Never silent thresholds.
- Select rows duplicating training rows → contamination error before any
  certificate is issued.

## What zugzwang is NOT

- It does not train engines (LightGBM/XGBoost are optional extras, never
  dependencies).
- It does not learn uncertainty scores — nothing learned on top of the
  model's own confidence paid, measured three ways in the trilogy behind
  this product. Bring a custom score as a callable if you want.
- It does not detect distribution drift. The production monitor (v1.0)
  watches observed coverage/risk against the certificate — measurable
  quantities — and alarms on deviation, with no epistemic claims.

## Install

```
pip install zugzwang          # coming with v0.1.0
```

Core dependencies: numpy, scipy, scikit-learn. Docs: quickstart, tutorials
(sklearn / LightGBM / raw scores), theory per guarantee, and the benchmark
pages — see `docs/` (GitHub Pages once public).

## Phases

| Phase | Deliverable | Release | Status |
|---|---|---|---|
| **F0** | Packaging, CI, frozen benchmark data | — | ✅ |
| **F1** | Certified core + `SelectiveClassifier` + contract B1–B4 run and published | v0.1 → PyPI | ✅ |
| **F2** | Opt-in calibration + self-contained HTML report | v0.2 | ✅ |
| **F3** | `DecisionGate` + monitoring | v1.0 | ✅ |

The companion book (Spanish): [`book/libro.pdf`](book/libro.pdf).

## License

MIT
