Metadata-Version: 2.4
Name: certlim
Version: 0.1.1
Summary: Finite-evidence certification limits: what any Certify/Reject rule can achieve from the evidence you have.
License-Expression: MIT
Keywords: certification,safety,partial identification,hypothesis testing,selective prediction,reliability,acceptance sampling,industrial informatics
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Manufacturing
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: scipy>=1.10
Provides-Extra: test
Requires-Dist: pytest>=7.0; extra == "test"
Dynamic: license-file

# certlim

**What any Certify / Reject rule can achieve from the evidence you actually have.**

`certlim` answers a question that accuracy metrics cannot: given two registered
classes of systems — those that are safe under a proposed change and those that
are unsafe — and the measurement you can afford to take, *how often can any
procedure return a reliable verdict while controlling both kinds of error?*

And when the answer is "not often enough", it tells you which of the two
remedies applies:

- **buy more of the same evidence** — the obstruction is temporary and closes at
  a computable block length; or
- **change the measurement** — the obstruction is permanent, and no amount of
  the same channel, nor any re-encoding of it, will ever help.

Distinguishing those two cases is the point. They look identical to a
single-sample detector and they cost very different amounts of money.

```bash
pip install certlim
```

---

## Sixty seconds

```python
import certlim as cl

# Two safe populations and one unsafe one, over a two-symbol evidence alphabet.
exp = cl.Experiment(safe=[[0.25, 0.75], [0.75, 0.25]],
                    unsafe=[[0.50, 0.50]])

d = exp.diagnose()
print(d.region)            # 'delayed'
print(d.n_star)            # 2
print(d.explain())
```

> Delayed separation: the one-unit hulls intersect but no exact pair collides,
> so repetition alone opens the gate at n = 2. Buying more of the same evidence
> is the correct move here.

Note what just happened. Every safe/unsafe **pair** in this example is well
separated — the minimum pairwise total variation is `0.25` — yet one unit of
evidence certifies nothing at all:

```python
exp.pairwise_min_tv()      # 0.25   "looks separable"
exp.delta(1)               # 0.0    actually is not
exp.decisiveness(1)        # 0.10 = alpha + beta, i.e. no better than a coin
exp.delta(2)               # 0.125  two units break the tie
```

A criterion built on pairwise distances would have declared success here. The
composite geometry is what governs, and it is what `certlim` computes.

---

## The frontier

```python
f = cl.frontier(exp, n_grid=[1, 2, 3, 5, 10], alpha=0.05, beta=0.05)
print(f)
```

```
   n   states     Delta         D     bound     slack  gate
   1        2  0.000000  0.100000  0.100000  0.000000     1
   2        3  0.125000  0.112500  0.225000  0.112500     0
   3        4  0.187500  0.116667  0.287500  0.170833     0
   5        6  0.273438  0.138889  0.373438  0.234549     0
  10       11  0.409561  0.196970  0.509561  0.312591     0
```

`Delta` is the composite separation, `D` the best worst-case decisiveness any
valid rule attains, `bound` the ceiling `min(1, alpha + beta + Delta)`, and
`gate` flags block lengths where the two hulls still meet.

`f.units_for(0.9)` returns the smallest gridded block reaching 90 % decisiveness,
or `None` if the grid does not get there.

---

## The deployable rule

```python
rule = cl.build_rule(exp, n=3, alpha=0.05, beta=0.05)
rule.decisiveness          # what it actually attains
rule.alert_rate_floor()    # the fraction of blocks that must come back Alert

decision = rule.decide([2, 1], rng=np.random.default_rng(7))
decision.outcome           # 'certify' | 'alert' | 'reject'

rule.error_profile(exp)    # realised worst-case side errors, self-check
```

The optimal rule is genuinely randomised, so pass a seeded generator whenever the
verdict has to be reproducible — in a certification workflow it usually does.

When the hulls still intersect, the optimum is a rule that **ignores the data**
and answers at the allowed error rates. `certlim` says so plainly rather than
dressing it up. That is the signal to change the measurement.

---

## From raw measurements

Start from per-unit symbol counts, one row per physical unit:

```python
exp = cl.Experiment.from_counts(safe_counts, unsafe_counts, smoothing=0.5)
```

Building the evidence menu is where most mistakes happen, so the quantiser
enforces the one property that makes menus comparable:

```python
q = cl.NestedQuantizer.from_quantiles(features, bins=2, drop_order=(2, 1))
q.chain()                  # [8, 4, 2] — strictly nested, by construction
counts = q.counts(features, level=0)
```

Quantising each menu independently produces partitions that are **not** nested,
and the resulting comparison can show a coarse menu beating its own refinement.
That is a pipeline artefact. `cl.check_data_processing(fine, coarse)` will tell
you when you have hit it.

---

## Finite-sample calibration

A plug-in `Delta` from observed counts is an estimate:

```python
cl.permutation_test(unit_counts, labels, blocks=chemistry, n_permutations=999)
cl.bootstrap_delta(unit_counts, labels, n_resamples=2000)
```

Both resample **whole physical units**. Resampling rows, cycles or time points
instead will understate the interval and inflate significance by orders of
magnitude, because repeated measurements of one unit are not independent
evidence about the population of units.

Neither is a lower confidence bound on the population separation. A significant
permutation p-value rejects label exchangeability; it does not certify anything
with coverage. `certlim` keeps that distinction in the result objects.

---

## Deployment and procurement

```python
report = cl.transport_margin(source_exp, deployment_exp)
report.certified           # separation survives the registered shift?
```

The margin is `Delta_source - drift_safe - drift_unsafe`, measured as class
geometry rather than point-prediction error. A non-positive margin means the
source evidence does not certify transport — not that the deployment domain
collides.

```python
cl.screen([
    cl.Plan("passive",   n=1, decisiveness=0.63, cost={"seconds": 780, "c_rate": 0.0}),
    cl.Plan("active",    n=1, decisiveness=0.53, cost={"seconds": 940, "c_rate": 0.5}),
    cl.Plan("nameplate", n=6, decisiveness=0.10, cost={"seconds": 0,   "c_rate": 0.0}),
], target=0.4)
```

Costs stay a vector and are never collapsed into one currency. A plan is dropped
only when another is no worse on every recorded axis. Plans measured on
different ledgers raise rather than being ranked — an unmeasured cost is not a
free cost.

---

## The report

One command turns the whole analysis into a filing-ready page:

```bash
certlim report units.csv -o certification.html --subject "Lot 2026-08"
```

```python
cl.write_report("certification.html", exp,
                unit_counts=counts, labels=labels, blocks=site,
                subject="Lot 2026-08",
                notes=("Grades come from the bench protocol, not from this channel.",))
```

The page carries the structural verdict and the action it implies, the frontier
as a table and an inline chart, the finite-sample calibration, the decision table
and the standing scope caveats. It is a single file with no external stylesheet,
script, font or image, so it survives being archived or opened offline, and it is
byte-deterministic unless you pass a `timestamp` — two runs of the same analysis
produce the same file, and a diff means something actually changed.

## Command line

```bash
certlim diagnose  units.csv                          # permanent, delayed, or open?
certlim frontier  units.csv --n 1 2 3 5 10 --target 0.9
certlim calibrate units.csv --block-col chemistry
certlim rule      units.csv --n 3 --json rule.json
certlim report    units.csv -o certification.html
```

Input is either a CSV of per-unit counts with a `grade` column holding `safe` /
`unsafe`, or a JSON file of probability laws:

```json
{"safe": [[0.25, 0.75], [0.75, 0.25]], "unsafe": [[0.5, 0.5]]}
```

Add `--json out.json` to any subcommand for a machine-readable report.

---

## What certlim does not do

It computes population geometry exactly. It does **not** know whether:

- your two classes were registered before you looked at the data;
- your grades were identified by a reference experiment rather than by the
  channel you are evaluating — grading units with the channel under test makes
  every number here meaningless;
- your sampling unit is the physical unit it should be;
- your evidence is conditionally i.i.d. within a block;
- your registered models are cohorts rather than individual units. If each model
  is one physical unit, two units can produce identical histograms by chance,
  which is an exact collision by definition. `certlim` flags that case rather
  than letting it read as a structural obstruction, but it cannot decide for you
  how the classes should have been registered.

Those are design decisions and the package will not make them for you. Every
result is relative to the registered classes: enlarging them can only shrink the
separation, never grow it.

---

## Method notes

Block laws are enumerated over count vectors, the sufficient statistic for i.i.d.
draws from a finite alphabet, so the problem is `C(n+K-1, K-1)` rather than
`K**n`. The composite distance and the optimal decision rule are solved as two
**independent** linear programs, so their agreement with the certification bound
is a real check rather than a same-code identity. There is no sampling, no
fitting and no tuning parameter anywhere in the core.

`delta()` is solved through the dual separation game, which is 10x to 450x
faster than the primal hull program on large state spaces. The dual loses
accuracy as the space grows, and a value near zero is exactly where the gate
verdict turns, so anything below `1e-5` is re-solved with the exact program;
`delta(n, exact=True)` forces it everywhere.

Exact enumeration has a ceiling (`max_states`, default 20 000). It is a **time**
budget, not a memory one — measured cost is about 3 s at 6 000 states, 40-100 s
at 20 000, and hours beyond, with run-to-run variation of a factor of two at the
same size. Exceeding it raises with the offending `(K, n)` and what to do about
it, rather than silently approximating or starting a job that never finishes.

For a finite registered class with `r` distinct one-unit laws and no exact
collision, the gate must be open by `r - 1`; `Experiment.stopping_bound()`
returns it, and `n_star()` searches exhaustively up to it. The bound is tight for
small classes and loose for large ones, and it bounds the *structural* search
only — never the number of physical units needed for a confidence statement.

## Licence

MIT.
