Metadata-Version: 2.4
Name: fishersketch
Version: 0.2.0
Summary: Vocabulary-scale output-error task signatures: a portable .fsig registry + a regime diagnostic for transfer source selection
Project-URL: Homepage, https://github.com/Sideplane/fishersketch
Project-URL: Repository, https://github.com/Sideplane/fishersketch
Project-URL: Paper, https://arxiv.org/abs/2606.27242
Author-email: John Sweeney <contact@sideplane.ai>
Maintainer-email: Sideplane AI <contact@sideplane.ai>
License: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: deduplication,fisher-information,llm,model-fingerprint,privacy-preserving,sketching,source-selection,task-registry,task-similarity,transfer-learning
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Requires-Dist: numpy>=1.21
Requires-Dist: torch>=2.0
Provides-Extra: all
Requires-Dist: numba>=0.58; extra == 'all'
Provides-Extra: examples
Requires-Dist: transformers>=4.30; extra == 'examples'
Provides-Extra: fast
Requires-Dist: numba>=0.58; extra == 'fast'
Provides-Extra: test
Requires-Dist: pytest>=7.0; extra == 'test'
Description-Content-Type: text/markdown

# FisherSketch

**Fingerprint a task in 192 KB — then compare, deduplicate, diagnose, and route across a registry
of tasks, without re-running the model and without sharing the data.** A FisherSketch signature
(`.fsig`) is an **aggregate** sketch of a task's training-error geometry — a pooled sum over the whole
dataset, with no per-example records — so you can exchange a task fingerprint where you could never
share the dataset (see **Privacy scope** below for exactly what it does and doesn't hide). And it works at **vocabulary scale (K ≈ 128k)** — where the classical
label-aware transferability scores (LEEP, LogME, H-Score) **cannot run at all**: the exact object is
a per-task K×K error covariance (~6 TB across 100 tasks); the 192 KB sketch is a **~1,510×**
reduction (LEEP's joint table alone is 128k² ≈ 10¹⁰ entries; LogME / H-Score degenerate at K≫N).

**For** teams maintaining many fine-tuning datasets or task variants on a **shared** model — dedup
near-identical tasks, find related ones, diagnose where transfer signal lives, and route to the best
source — **with no model and no raw data on hand.** Straight from the shell:

```bash
python -m fishersketch inspect sigs/    # what's in a registry of .fsig fingerprints (no model needed)
```

**Share fingerprints, not data.** This is the differentiator: a registry of `.fsig` files carries a
corpus's *update geometry* without the corpus — something an embedding-and-cosine workflow can't
cleanly give you.

**What it is — and isn't.** It *is* a fixed-model task **registry** + a regime **diagnostic** + the
only label-aware task signature that runs at vocabulary scale; mechanically, the signature is the
task's output-error (Fisher) update geometry, sketched in one streaming pass. It is **not** a general
transfer ranker — when activations are informative a cheap class-conditional centroid can match it
on source-ranking, and the activation×error coupling is a ranking wash. Its edge is **feasibility
where nothing else runs, the privacy-preserving registry, and the activation-dark floor**: where a
verbalizer/format shift leaves representation methods at chance (provably blind — Theorem 3.2), the
error geometry still recovers transfer (≈3× over random).

Reference: Sweeney, *The Geometry of Updates: Fisher Alignment at Vocabulary Scale*,
ICML 2026 — [arXiv:2606.27242](https://arxiv.org/abs/2606.27242) (PMLR v306, to appear).

## When to reach for this (and when not)

Two questions decide whether any transferability method works for **shared-vocabulary LLM
source selection** — and every prior method fails one of them:

**1. Does it see the *error* geometry, not just activations?**
Representation-similarity metrics — **CKA, RSA, SVCCA** (and the activation-only second
moment `Scov(Ma)`) — read only the probe activations. They are *provably* blind to the head
update: identical activations can hide orthogonal updates (Theorem 3.2). In the
**activation-dark** regime — e.g. a verbalizer shift, where you change only the answer token
so the answer-position activation literally cannot move — they **collapse to random**.

**2. Does it scale to vocabulary K ≈ 128k?**
Classical transferability scores — **LEEP, LogME, H-Score** — are label-aware, so they
*don't* collapse in activation-dark. But they hit a **vocabulary-scale wall**: **LogME and
H-Score degenerate when K ≫ N** (you can't estimate per-class structure over 128k classes
from a probe of a few hundred samples), and **LEEP hits a K² memory wall** (its 128k×128k
joint label table ≈ 10¹⁰ entries). They can't run where you operate.

FisherSketch is the one of these methods that answers **yes to both** — it scores the joint
(activation, **error**) update geometry *and* sketches it in one streaming pass at vocabulary
scale. Measured on Llama-3.1-8B, it has the best **worst-case across regimes**:

| Method | Natural shift | Verbalizer (activation-dark) | **Worst case** |
|---|---:|---:|---:|
| `Scov(Ma)` / CKA-family | 46.3% | 20.0% (= random) | 20.0% |
| `Scov(Γe)` error-only | 41.3% | 72.2% | 41.3% |
| **FisherSketch** | 45.7% | 66.7% | **45.7%** |
| LogME / H-Score | — | — | degenerate (K≫N) |
| LEEP | — | — | K² memory wall |

The cheap competitive option is **error-only `Scov(Γe)`** — it scales and sees the errors,
and is in fact *better* than the joint in the activation-dark column (72.2% vs 66.7%), but it
has a worse **worst-case** (41.3%) because it ignores the activation geometry that helps under
natural shift. FisherSketch's edge here is **feasibility and the activation-dark floor**, not a per-regime
ranking win: it is the one of these that runs at K≈128k *and* sees the error geometry, so it is
never catastrophic across regimes (the worst-case column). The activation×error
**coupling** `ρ` itself adds no measured ranking value — `regime()` reports `coupling: engaged`
vs `~squashed`, and when squashed the joint simply tracks the error marginal; within a single
regime a specialized method (error-only when activations are dark, a class-conditional centroid
when they're informative) can out-rank the joint on *ranking*. That comparison flips on
*relatedness / retrieval* — the registry's main job: the FisherSketch fingerprint edges a plain
activation centroid by **+0.038 micro-accuracy** (m=4096, paired p<0.001) on the paper's 700-task
NaturalInstructions retrieval. A modest, **no-label** margin — when task labels *are* available a
learned linear probe beats the fingerprint on the same retrieval, so reach for this where you can't
train a probe. Measure on your own data. The bundled
`fishersketch.evaluation` baselines (`cka`, `logme`, `hscore`, `leep`, and the `Scov`
marginals) let you reproduce each collapse on your own data. Call `scalability_report(d, n, K)` to
surface the K≫N / K² wall directly — a raw `logme()` / `hscore()` call below its class ceiling
returns a plausible-but-meaningless finite value rather than erroring.

**Decision flow:** shared output basis? → activations activation-dark? → stream FisherSketch
→ rank sources + audit ρ.
**Best fit:** shared-vocab LLM families and scientific string domains (SMILES, protein,
genomic tokens). **Out of scope:** heterogeneous outputs — use the full-network sketch
(`FisherNetwork`). It is **checkpoint-conditional** (Fisher at one checkpoint) and one signal
among many — pair it with coverage / fairness / safety before automating.

## What's in this build

The full validated pipeline behind the paper's numbers:

- **Factored Random-Maclaurin** features for the degree-2 product kernel `(a·a')²(e·e')²`,
  `O(m(d+K))` per sample.
- **SRHT** projections for the K-dimensional error, **auto-enabled at vocabulary scale**
  (`dim_e ≥ srht_threshold`, default 65536) via a Walsh–Hadamard transform —
  `O(K log K)` instead of `O(mK)`. numba-accelerated when the `fast` extra is installed,
  with a vectorized numpy fallback otherwise.
- **Split-half (A/B) estimation**, **U-statistic** diagonal correction, **positivity
  clamping**, the **χ/ρ coupling** decomposition, and **empirical-Bayes ρ-shrinkage**
  (the robust vocabulary-scale ranking estimator).

## Install

```bash
pip install fishersketch              # core: numpy + torch
pip install "fishersketch[fast]"      # + numba (JIT-accelerated CPU SRHT; optional)
pip install "fishersketch[examples]"  # + transformers (only the AI4Sci example + HF benchmarks)
```

From a clone, for development: `pip install -e ".[test]"` (adds pytest).

**torch is required** — it is the array backend and the GPU engine. It runs on CPU by
default (`device="cpu"`); pass `device="cuda"` or `"mps"` to accelerate. numba is
**optional**: without it the SRHT Walsh–Hadamard transform falls back to a vectorized
numpy implementation (identical results, slower at vocabulary scale).

**First result in 60 seconds (no model):**

```bash
pip install fishersketch
python -m fishersketch.evaluation                 # activation-dark surrogate: error-geometry 0.83 vs activation-only 0.17
# from a clone, the full registry walkthrough:
python examples/00_atlas_registry_quickstart.py   # build -> save .fsig -> reload registry -> diagnose -> recommend
```

Examples are tagged: `00` / `02` / `04` run as-is; `01` / `03` / `05` need a torch model.

## Quickstart (copy-paste runnable)

No model needed — `fishersketch.demo` ships tiny synthetic streams so this block runs as-is:

```python
from fishersketch import FisherAtlas
from fishersketch.demo import make_toy_streams

# {name: (a, e)} with a:(n, d) activations and e:(n, K) errors (= softmax(logits) - onehot(y)).
# The toy tasks share an activation backbone (activation-dark) and split into two error groups,
# so there is a real ranking to recover.
streams = make_toy_streams()

atlas = FisherAtlas.from_streams(streams, sketch_dim=2048, seed=0)
rec = atlas.recommend("task0")          # which other task best transfers to task0?
print(rec.explain())                    # best source, score, gap to runner-up, confidence
```

With no model at all, `python -m fishersketch.evaluation` runs the activation-dark surrogate — a
synthetic check that the error geometry recovers transfer (~0.83 top-1) where activation-only sits
near chance (~0.17).

Runnable scripts are in [`examples/`](examples/): a model-free **atlas → registry → diagnostic**
quickstart (`00`, no assets needed), source selection on a model (`01`), signature persistence
(`02`), the full network + certificate (`03`), evaluating against the baselines on your own data
(`04`), and AI-for-science masked-LM streams (`05`). `00` / `02` / `04` run as-is; `01` / `03` /
`05` need a torch model.

### One call, straight from a model

If you have a torch model and per-domain dataloaders, skip the manual extraction entirely:

```python
from fishersketch import recommend_sources

rec = recommend_sources(
    model,
    target="my_domain",
    candidates=["wiki", "code", "legal"],
    data={...},                         # {name: dataloader} yielding (inputs, labels)
    head=model.lm_head,                 # the head whose Fisher alignment you want
    task="lm",                          # "lm" | "classification"
)
print(rec.best.name, "->", rec.explain())
```

`recommend_sources` hooks the head, extracts `(activation, error)` for each domain in one
streaming pass, builds the atlas, and returns a `Recommendation` (best source, per-source
scores, the gap to the runner-up, and a confidence flag).

### Reading the recommendation

`recommend()` defaults to **`method="auto"`**: it first reads the **regime** — which
geometry actually carries the transfer signal — and then scores by that geometry rather
than blindly returning the joint. `rec.explain()` prints the pick, the regime it followed,
and the signals that tell you whether to trust it (below, the bundled `make_toy_streams()`
demo — an activation-dark task set, at `sketch_dim=2048, seed=0`):

```
Recommended source for 'task0': task1
  regime: activation-dark -> auto-scored by 'error_only'  (activation 0.005, error 0.955, coupling alpha 0.00)
  task1            A_F=+1.111  (|src|/|tgt|=1.01)  <- best
  task3            A_F=+0.168  (|src|/|tgt|=0.99)
  task2            A_F=+0.159  (|src|/|tgt|=0.99)
  verdict: LIKELY worth it  (#1-vs-#2 gap 2.1 cand-sigma; vs random +1.4 sigma; winner se(logrho)=0.04)
  baselines: activation-only picks 'task2'; error-only agrees
  note: activation geometry is ~constant across sources (activation-dark / verbalizer regime -- error geometry is doing the work)
  coupling: ~squashed (alpha=0.00) -- at this sample count the ranking tracks the marginal product; collect more samples to engage the coupling term
```

This is the representative case: the coupling reads `~squashed` (engaging it needs hundreds of
samples per task), and the activation-dark note shows the error geometry carrying the signal — the
regime FisherSketch is built for.

The score is the **symmetric `A_F`** (the directionless similarity); `recommend(..., directed=True)`
opts into the directed coefficient `B = A_F·|i|/|j|` (shown as `B=…`), whose `|i|/|j|` norm ratio is
data-dependent. The `regime:` line is the key control. Each geometry gets an **engagement** number (the
symmetric analog of the coupling's α); a degenerate one can't rank, so `auto` drops it:

| Regime (`atlas.regime()`) | What it means | What `auto` scores by |
|---|---|---|
| `coupled` | activation **and** error informative, coupling engaged | the joint `fisher` score |
| `separable` | both informative but coupling squashed (joint tracks the marginal product) | `fisher` (still orients the ranking) |
| `activation-dark` | activations don't separate the tasks (verbalizer shift, small-scale SMILES) | **`error_only`** — the detector defers the joint where the activation factor is below engagement |
| `error-flat` | errors ~uniform across tasks | `activation_only` |
| `no-signal` | neither geometry separates the tasks | `fisher`, but treat any pick as low-confidence |

The remaining lines refine the trust call:

| Line | What it tells you | What to do |
|---|---|---|
| `coupling: engaged (alpha≥0.15)` | both geometries are informative and not independent — the score reflects both, not a single marginal | **the pick is supported by both geometries** (subject to the verdict) |
| `coupling: ~squashed (alpha≈0)` | the joint tracks the marginal product at this sample count — the coupling is below engagement | gather more samples (~256+/domain) to engage the coupling |
| `power: LOW` (shown when present) | the winning pair is statistically noisy | gather more samples before trusting it |
| `verdict: NO source clears the bar` | nothing is strongly aligned | don't transfer |
| `baselines:` | the free cross-check | error-only **agrees** → solid ground; if the joint **disagrees** with both marginals, treat the pick with caution — the cheap marginal is the safer signal |

The verdict is gated on the coupling: it will **never** say `CONFIDENT` when the coupling is
squashed (a low per-pair SE can't rescue a globally-uninformative coupling).

### Scale to vocabulary (the real regime)

For an LLM head you build the atlas incrementally and let SRHT auto-engage:

```python
# d = hidden size entering the head; K = vocab/output size.
atlas = FisherAtlas(dim_a=4096, dim_e=128256, sketch_dim=4096, device="cpu")

for name, (acts, errs) in task_streams.items():     # acts:(n, d), errs:(n, K)
    atlas.add_task(name, acts, errs)                # error = softmax(logits) - onehot(y)

names, A = atlas.alignment(shrinkage=True)          # full T×T alignment matrix
diag = atlas.diagnostics()        # S_ae, S_a, S_e, chi, rho, rho_shrunk, fisher_shrunk, ...
print("SRHT active:", atlas.uses_srht)              # auto-on once dim_e >= srht_threshold
```

### AI-for-science (scientific foundation models)

Scientific foundation models are first-class. **Masked-LM encoders** — ESM-2 (proteins),
ChemBERTa (molecules), DNABERT / Nucleotide Transformer (genomics) — go through
`fishersketch.extract.mlm_streams`, which masks tokens and captures the MLM head-Fisher
signal `(a, e)` for you; **causal scientific LMs** (ProtGPT2, ProGen) use the same
`recommend_sources(..., task="lm")` path as any LM. See **`examples/05_ai4sci_masked_lm.py`**
(and `benchmarks/sci_data.py` / `moleculenet_data.py` for UniProt / MoleculeNet / genomic
loaders to bring your own corpora).

```python
from fishersketch.extract import mlm_streams
streams = mlm_streams(esm_model, esm_tokenizer, {name: [seq, ...] for name in corpora})
atlas = fishersketch.FisherAtlas.from_streams(streams)
print(atlas.regime().explain())     # which geometry carries the signal on YOUR data?
rec = atlas.recommend(target)       # ranks sources by the geometry the diagnostic trusts
```

The honest value here is **diagnosis, not a magic ranker**. The `regime()` readout tells you
*where* transfer signal lives for your model + data — and that answer is informative: a
domain-native encoder (e.g. ESM) carries family structure in its **activations** while the
per-token errors are near-flat (all proteins share one amino-acid alphabet), so the detector
routes to the activation geometry — a distinction a token-frequency baseline cannot make. The
flip side, stated plainly: on a **diffuse general LM** applied to scientific strings the per-token
error can be dominated by the token-frequency term (high next-token entropy → the `−onehot(y)` term
dominates), and the detector will tell you so (`error-flat` / squashed coupling) rather than
quietly hand you token statistics dressed as geometry. Use a domain-native scientific FM, and
read the diagnostic before trusting any ranking.

## Persist & share signatures (`.fsig`)

Sketch each domain once, save it, and compare new domains against the saved set later —
no need to re-run the source models. A signature is the six unnormalized `{A,B}×{ae,a,e}`
accumulators (192 KB at `m=4096`, 48 KB at `m=1024`); reloading and replaying them
reproduces the live atlas's `fisher_shrunk` / `directed_fisher` **bit-exactly**. A `.fsig` is a
versioned `FSIG` binary (magic + JSON header + float64 payload) — load it with
`FisherSignature.load(path)` or `SignatureStore.from_dir(dir)`, not `np.load`.

```python
from fishersketch import FisherAtlas, SignatureStore

# Producer: sketch domains (maybe on different machines) and write one .fsig each.
FisherAtlas.from_streams(streams, sketch_dim=4096, seed=0).save_signatures("sigs/")

# Consumer: load the open set and score it (shrinkage is set-dependent, so adding a
# signature can move other pairs — by design).
store = SignatureStore.from_dir("sigs/")
print(store.recommend("my_target").explain())
names, A = store.alignment()
```

A `SignatureSpec.key()` fingerprint **hard-gates** every comparison: signatures built in
different projection bases (seed, dims, sketch method, `augment_bias`, …) raise
`IncompatibleSignatureError` naming the offending field, rather than returning silent
garbage. `compare(sig_i, sig_j)` scores a single pair; `sig_a + sig_b` merges two shards
of the *same* domain collected separately.

**From the shell.** No script needed — `python -m fishersketch` works over a directory of `.fsig`:

```bash
python -m fishersketch inspect  sigs/            # table: tasks, samples, grad-norm, model, basis key
python -m fishersketch diagnose sigs/            # which geometry carries the signal (regime)
python -m fishersketch dedup    sigs/            # near-duplicate task pairs (self-normalized cosine)
python -m fishersketch nearest  sigs/ my_task    # the tasks most related to my_task
python -m fishersketch score    sigs/ my_target  # rank sources (symmetric A_F; --directed opts in to |i|/|j|)
```

`inspect`/`dedup`/`nearest` read only metadata + the joint vectors (no model); `diagnose`/`score`
replay into the estimator. In Python, `FisherRegistry` aliases `SignatureStore`, and
`score_registry(dir_or_store, target)` is the one-call score. Each subcommand has its own `--help`
(e.g. `python -m fishersketch score --help`): `score --method/--directed/--candidates`,
`dedup --threshold`, `nearest -k`.

### Privacy scope — what a `.fsig` does and doesn't hide

A `.fsig` lets you exchange a task fingerprint without shipping the dataset, but be precise about the
guarantee — it is **aggregation-based obfuscation, not differential privacy** (no ε, no formal proof).
Against an adversary holding the `.fsig`, the published projection seed, and the model:

- **Obscured** (when many samples are pooled): the per-example activations and labels, and the *signed*
  label-mean direction. The degree-2 sketch is sign-blind and the signature is a pooled sum, so
  individual records and the update's sign aren't directly readable. This **weakens as the pool
  shrinks** — at `N=1` the sum *is* the single example, and given the model a small-`N` label set can
  be brute-forced.
- **Recoverable by construction**: the aggregate second-moment (activation/error) covariance — that is
  literally the statistic the sketch computes, so anyone with the `.fsig` can read it.

Pool a meaningful number of samples per task (the same `n ≳ 32` the estimator wants for a stable
score), and treat a `.fsig` as "doesn't carry the raw corpus," not as a cryptographic guarantee. For a
formal bound, add noise under a DP mechanism upstream — FisherSketch does not.

## Beyond the head: multi-layer transfer (`FisherNetwork`)

The head estimator scores the last layer's update. `FisherNetwork` is the real **Full
FisherSketch** (Theorem O.18): it owns forward/backward, hooks every tracked `nn.Linear`,
and sketches the **concatenated** per-layer Kronecker gradient `g = [a₀⊗δ₁; …; a_{L-1}⊗δ_L]`.
The exact kernel sums *inside* the square — `(g·g')² = (Σ_l (a_l·a_l')(δ_l·δ_l'))²` — so it
is **cross-layer aware**, unlike the per-layer "block" proxy that drops the interaction.

```python
from fishersketch import FisherNetwork

net = FisherNetwork(model, layers="auto", sketch_dim=4096)   # all nn.Linear
with net.capture():
    for name, loader in data.items():
        net.begin_task(name)
        for inputs, labels in loader:
            model.zero_grad()
            loss_fn(model(inputs), labels).backward()        # your loss
        net.end_task()

names, A = net.alignment(mode="full")    # "full" | "block" | "off" (cross-layer only)
print(net.recommend("my_target").explain())
print(net.certificate())                 # is the cheap block answer trustworthy here?
```

The one-pass decomposition is `A_full = A_blk + A_off` (block + cross-layer interaction,
disjoint support). `certificate()` returns a profile cosine and a certified bound
`|cos_full − cos_blk| ≤ (1−c_r) + w_off·|cos_off − cos_blk|` — it tells you *when* the cheap
coupling-blind block answer can be trusted (the paper sees `c_r ≥ 0.90`). Full-network
signatures persist as `estimator="full"` `.fsig` and replay bit-exactly, exactly like the
head. v1 tracks `nn.Linear` only (incl. tied `lm_head`); disable activation checkpointing on
tracked layers.

## Power-user API

Most users only need `FisherAtlas` (or `FisherNetwork`). The validated estimators and the
lower-level sketchers live in `fishersketch.backend`:

```python
from fishersketch.backend import (
    FisherSketchGPU,            # the torch engine FisherAtlas drives
    FisherSketch,              # numpy CPU estimator
    SeparableLayerProxyGPU,    # the coupling-blind per-layer proxy (Σ_l S_a·S_e)
)
```

`SeparableLayerProxyGPU` (formerly `BlockDiagonalProxyGPU`, kept one release as a deprecated
alias) registers hooks on `nn.Linear` modules and sums the matched per-layer
`Scov(Mₐ)·Scov(Γₑ)` products. It is the **coupling-blind** proxy: it drops both within-layer
a–e coupling *and* cross-layer interaction. For the coupling-aware multi-layer estimator use
`FisherNetwork`; for the coupling-aware head, `FisherAtlas`.

## Estimator semantics & caveats

- **`alignment()` / `rank_sources()` return a RANKING score** (higher = a better predicted
  transfer source; the diagonal is ~1). It is the U-statistic estimate of the head Fisher
  alignment — well-calibrated to the population value (the cosine of the task mean
  embeddings) — but, being an unbiased ratio rather than a Cauchy–Schwarz-bounded cosine,
  it can slightly **exceed 1** for highly similar tasks (an overshoot, like an unbiased
  correlation). It is intentionally **not clipped** to `[0, 1]`: clipping flattens the
  graded values into a constant tie and destroys the ranking in exactly the shared-error /
  activation-dark regime this library targets. Orthogonal tasks read ~0. The default
  `shrinkage=True` adds empirical-Bayes ρ-shrinkage (ρ capped at the separability ceiling 1
  to bound the near-orthogonal 0/0 ratio — ranking-safe, slightly compresses genuine
  super-coupling); `shrinkage=False` is the raw joint cosine — noisier, and positively
  biased on near-orthogonal pairs.
- **It is not the literal Algorithm-1 pooled-μ cosine.** It is the split-half (A/B)
  estimator with a per-split U-statistic diagonal correction and positivity clamping
  (paper Sec 5.3 / App B); both are unbiased for the same population.
- **Input scale.** The estimator is invariant to rescaling the errors, but it assumes
  the documented input `e = softmax(logits) − onehot(y)` (`‖e‖² ≈ 1`). Feed the raw head
  error, not a normalized or per-sample-rescaled version (per-sample normalization
  changes the kernel). NaN/inf inputs are rejected; each task needs **≥ 2 samples**
  (≥ 4 recommended) for the A/B split.
- **Backends.** `FisherAtlas` always uses the torch `FisherSketchGPU`. The numpy
  `FisherSketch` is a lighter CPU reference with a less numerically-stable χ/ρ path; the
  two are not bit-identical. On MPS, accumulators fall back to float32 (96 KB per-task
  state vs. 192 KB), slightly reducing precision at large sample counts.
- **Evidence — what is checked where.** *Method:* the kernel identities, sketch unbiasedness,
  SRHT, and the non-identifiability theorem (Thm 3.2) are unit-tested in `tests/`. *Demonstration:*
  `python -m fishersketch.evaluation` runs the activation-dark surrogate and prints error-geometry
  0.83 vs activation-only 0.17 — synthetic, no model. *Paper:* the real-model numbers — the ~72%
  activation-dark top-1 and the ~1,510× memory figure — are from Llama-3.1-8B with real
  activation/error streams; they are **not** reproducible from this repo, which ships no weights.
  Bring your own streams to `fishersketch.evaluation.run_eval`. This is an early (0.2.0) release.

## Troubleshooting

| Symptom | Cause | Fix |
|---|---|---|
| `IncompatibleSignatureError` on compare/add | signatures built in different bases (seed, dims, sketch method, `augment_bias`) | re-sketch with matching settings; a `.fsig` compares only within one basis **and** one base model |
| `SuspiciousErrorScaleWarning` | you passed raw logits, probabilities, or per-sample-normalized errors | pass `e = softmax(logits) − onehot(y)`, or use `fishersketch.extract` |
| a score above 1 (e.g. `compare` returns 2.6) | expected — the alignment is an unbiased U-statistic ratio, not a clipped cosine | use it for ranking/relatedness; the order is what's calibrated, not the magnitude |
| `ScalabilityWallError`, or `logme()`/`hscore()` returning a meaningless finite number | `K ≫ N` — the classical baseline can't estimate per-class structure | call `scalability_report(d, n, K)` to see the wall; this is the regime FisherSketch is for |
| `np.load` fails on a `.fsig` | `.fsig` is a versioned `FSIG` binary, not an npz | load with `FisherSignature.load(path)` or `SignatureStore.from_dir(dir)` |
| adding a signature moved other pairs' scores | the empirical-Bayes shrinkage is set-dependent, by design | for a pair-invariant score use `shrinkage=False`, or `nearest`/`duplicates` (already pair-invariant) |
| `SignatureStore(path)` raises `TypeError` | the constructor takes a `SignatureSpec`, not a path | use `SignatureStore.from_dir(path)` |

## API at a glance

| | |
|---|---|
| `FisherAtlas` | stream tasks (`from_streams` / `from_model` / `add_task`) → `alignment` / `rank_sources` / `recommend` / `regime` / `diagnostics` / `signature` / `save_signatures` |
| `SignatureStore` (alias `FisherRegistry`) | open-set registry: `from_dir` / `add` → `summary` / `regime` / `rank_sources` / `recommend` / `nearest` / `duplicates` |
| `FisherSignature` | portable `.fsig` artifact: `save` / `load`, `+` merges shards, `to_joint` → 16 KB `.fsigj` |
| `compare(a, b)` / `align(sigs)` | pairwise / cohort alignment from signatures |
| `recommend_sources(model, target, candidates, data)` | one call from a live model → `Recommendation` |
| `score_registry(dir_or_store, target)` | one call from a saved registry, no model → `Recommendation` |
| `classify_regime` / `RegimeDiagnostic` | which geometry carries the transfer signal |
| `FisherNetwork` | multi-layer (full-network) variant |
| `extract.head_streams` / `mlm_streams` | model → `(activation, error)` streams |
| `evaluation` / `baselines` | eval harness + CKA / LEEP / LogME / H-Score / centroid |
| `python -m fishersketch` | CLI: inspect / diagnose / dedup / nearest / score |

Inline type hints ship via `py.typed`.

## Status

`0.2.0` — full pipeline (validated against exact Fisher in `tests/`) plus the adoption
layer: one-call `recommend_sources`, the model→signals extractor, and portable `.fsig`
signatures with bit-exact replay. An early (pre-1.0) release; the public API may still
change before 1.0. Licensed under Apache-2.0 (see `LICENSE`/`NOTICE`); Copyright 2026 Sideplane AI.
