Metadata-Version: 2.4
Name: stoppower
Version: 0.1.0
Summary: Size your early-stopping window by statistical power instead of by habit
Author-email: Maximiliano Rodrigo Speranza <maximiliano.speranza@gmail.com>
License: MIT
Project-URL: Paper, https://doi.org/10.5281/zenodo.21630279
Project-URL: ORCID, https://orcid.org/0009-0005-0413-8554
Keywords: early-stopping,statistical-power,convergence,machine-learning
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# stoppower

[![tests](https://github.com/SperanzaMax/stoppower/actions/workflows/tests.yml/badge.svg)](https://github.com/SperanzaMax/stoppower/actions/workflows/tests.yml)
[![PyPI](https://img.shields.io/pypi/v/stoppower)](https://pypi.org/project/stoppower/)

Size your early-stopping window by statistical power instead of by habit.

*[Léeme en español](README.es.md)*

```bash
pip install stoppower
```

## The problem

Most people decide a model "converged" by comparing two or three evaluations and loosening the
tolerance when detection fails. That tunes the wrong knob. In the measured case
([Zenodo 10.5281/zenodo.21630279](https://doi.org/10.5281/zenodo.21630279)), the total detection
gain decomposes into **7.0× from lengthening the window** and only **1.31× from changing the
statistic**. Tolerance barely moves the needle.

Worse: once the signal-to-noise ratio drops below 1, **no** tolerance recovers the decision. Type I
and Type II error can no longer be separated by any choice of threshold.

## The rule

For `n` equally spaced evaluations, the standard error of the OLS slope decreases as `n^(-3/2)`,
against `n^(-1/2)` for plain averaging:

```
SE(b) = σ · sqrt( 12 / (n(n² − 1)) )
b*   ≥ (z_α + z_β) · SE(b)
```

## Usage

```python
from stoppower import window_for, sigma_from_pilot, evaluate, prereg_text, pts

# 1) What window do I need so I don't miss 1 accuracy point per 2500 steps?
plan = window_for(b_star=pts(1.0), sigma=0.0082, eval_every=500)
print(plan)      # -> n=11 evaluations = 5000 steps

# 2) How much power does the window I already use actually have?
from stoppower import power_of
power_of(pts(1.0), sigma=0.0082, n=3, eval_every=500)     # -> 0.097

# 3) Decide with data in hand
evaluate(steps, accs, b_star=pts(1.0))
```

```bash
stoppower design --b-star 0.01 --sigma 0.0082 --eval-every 500 --prereg
stoppower power  --b-star 0.01 --sigma 0.0082 --n 3 --eval-every 500
stoppower check  history.csv --b-star 0.01 --span 5000 10000
```

## A real case, end to end

Validation history of a model trained for 10,000 steps, evaluated every 500. The question is
whether the tail is still improving or it is safe to stop.

```bash
$ stoppower check history.csv --b-star 0.01 --span 5000 10000
  sigma estimated over the 5000-10000 span: 0.010593
UNDECIDABLE · slope +0.006424 per 2500 steps (95% CI [-0.003474, +0.01632])
  t=1.27 p=0.1017 · n=11 evaluations · power=0.63 for b*=0.01
  ! power 0.63 < 0.8: this window cannot support a convergence claim,
    only "no improvement was detected". Lengthen the window before concluding.
```

A two-point criterion would have said "converged". Here the verdict is **undecidable**, with the
number next to it: the window has no power to sustain that claim.

And if you hand it the whole run instead of the stable regime, it warns before answering:

```bash
$ stoppower check history.csv --b-star 0.01 --span 500 10000
  ! noise changes 7.4x within the window (sigma=0.06951 in the 1st half vs
    0.009344 in the 2nd, higher at the start): the equation assumes constant noise.
    Shorten the window or move it to the stable regime.
```

## Three things that set it apart from copying the formula

**1. σ is estimated over a span, not over the whole run.** Validation noise is not constant.
Measured across 8 seeds of the same model:

| step | 500 | 1000 | 1500 | 2000 | 2500 | 5000 | 7500 |
|---|---|---|---|---|---|---|---|
| SD across seeds | .101 | .073 | .038 | .010 | .014 | .009 | .011 |

A factor of 9. Feeding a global σ into the equation is off by a large factor, which is why the span
is a required argument here. `homoscedasticity_check` warns when the constant-noise assumption does
not even hold inside the window.

**2. It does not confuse "no improvement detected" with "converged".** `evaluate` returns
`converged=None` when power is insufficient, instead of a false green light. That asymmetry is the
whole point: failing to reject improvement is not evidence of convergence unless the study had the
power to see it.

**3. It emits the pre-registerable paragraph.** `prereg_text(plan)` produces the text with σ, its
provenance, the level, the power and the resulting window — to paste into your protocol **before**
looking at the data.

## Estimating σ without circularity

| situation | function |
|---|---|
| validation set is resampled each evaluation | `sigma_floor(p, n_val)` — analytic bound, no training needed |
| fixed set, you have a pilot run | `sigma_from_pilot(steps, values, span=(from, to))` |
| you already have several seeds | `sigma_from_runs(runs, at_index=...)` |

Fixing a window from a pilot is not the same as recalibrating a tolerance after seeing results: what
gets fixed is the **design**, not the verdict.

## Validation

The acceptance test reproduces Table 2 of the paper (σ = 0.00821, α = .05 one-sided, power = .80):
n=4 → b\* = 4.56, n=8 → 1.58, n=16 → 0.55 accuracy points per 2500 steps.

Beyond arithmetic, the nominal power was checked against a Monte Carlo of 4,000 simulations per
case: **empirical power 0.824** against 0.80 nominal, **empirical α 0.054** against 0.05, and
**0.092 for n=3** against the 0.097 the formula predicts. The rule is calibrated, not just
implemented.

## Scope

No dependencies (standard library only). It does not train, does not touch your loop, does not
decide for you. It does not model hardware effects: switching backends was measured to contribute
0.53× the across-seed variation in the late regime, negligible against what is already reported.

## Citation

Speranza, M. R. (2026). *Stopping criteria below the signal-to-noise floor: window length, not
tolerance, governs convergence detection in architecture comparisons.*
[10.5281/zenodo.21630279](https://doi.org/10.5281/zenodo.21630279)

MIT.
