Model updates, prompt edits, silent provider changes — none of them throw errors. dedrift catches the behavioral shift, tells you what moved and by how much, and points at the config change that did it.
LLM systems fail differently: outputs stay well-formed while length, refusals, tool use, latency tails, and meaning quietly shift.
The API alias stays the same. The model behind it doesn't. Nobody emails you.
A colleague "improves" the system prompt. Refusal behavior moves 8 points. No test fails.
New embeddings, new chunking, new retrieval order — same code, different agent.
Frozen canary prompts, run N times per cycle, compared as distributions — because single LLM runs are noise.
A versioned suite across six behavioral families: happy-path to adversarial.
Every record carries a config fingerprint of your model, prompt, and tools.
One primary test per channel → BH-FDR → materiality. Fewer, higher-confidence alerts.
"Behavior shifted 0.1 h after the model fingerprint change" — consistent with, never caused by.
Four cycles on one production model, then a silent switch to another — same prompt, same canaries.
Monitoring tools love p-values and hate publishing their false-alarm rates. We enforce ours in CI — a release cannot ship if the statistics fail their own audit.
The statistics are free forever. The advanced tier funds the work.
A commercial tier with advanced inference is in development — write to us to hear when it ships.
If a number carries a p-value, the simulation proving its false-alarm rate runs on every commit. A release that fails its own audit doesn't ship.
FDR-controlled significance and an effect worth acting on. Statistically real but practically trivial doesn't page you — alert fatigue kills monitoring.
Small suites have limited power, and the docs show that math instead of hiding it. Overclaiming is just drift of another kind.
Don't take our word for any of it — audit the statistics.