Core Concepts

Three ideas explain why Distil's numbers are real and why the approach is different: decision-equivalence as the correct quality target, a statistical certificate that enforces it, and cache-aware cost modeling that makes aggressive compression financially correct.

1. Decision-equivalence, not byte-equivalence

Most context compressors frame the problem as a string transformation: shrink the context, compare the output byte-by-byte, report overlap as accuracy. That framing has a fatal flaw — byte-equivalence and high compression are information-theoretically in tension. A claim of "87% compression, 100% accuracy" under byte-equivalence is self-contradicting; the two numbers cannot describe the same operation.

Decision-equivalence is the correct target. An agent only needs to take the same actions and produce the same outputs on the compressed context as it would on the original. "Same decision" means:

The same tool calls (name + arguments) in the same order.
The same final answer or action at each turn.
Measured across a real multi-turn trajectory, not a single prompt.

This is measurable, certifiable, and compatible with aggressive compression. Context blocks that are causally inert — they never change a decision when removed — are provably free to drop, regardless of how many bytes they contain. Distil's salience protection identifies the load-bearing frontier (model-free, using a causal ablation pass rather than a scorer model) and ensures those blocks are never touched, regardless of compression depth.

The reframe in one sentence: "100% accuracy" becomes a statistical non-inferiority guarantee on outcomes, not a diff of strings — and that makes it real. On a live claude-opus-4-8 grading run, Distil certified 83.2% token savings at a 0% decision-change rate.

2. The quality contract

A compression strategy in Distil does not ship unless a pre-registered statistical gate certifies it. Distil uses two complementary certification methods, both grounded in conformal prediction:

TOST (Two One-Sided Tests) — the standard non-inferiority framework used in pharmaceutical trials. Pre-register a margin (default: 2 pp drop in task-success rate) and significance level (default: α = 0.05). If the lower one-sided test rejects H₀ that mean ≤ −margin, the strategy is certified non-inferior.
DERC (Distribution-free Equivalence with Risk Control) — a Learn-Then-Test / CRC certificate that gives P(R(λ̂) ≤ α) ≥ 1−δ with Hoeffding–Bentkus p-values. No distributional assumptions; valid at finite sample sizes.

The Student-t tail used in TOST is computed from a hand-rolled regularised incomplete beta function — zero dependencies on scipy or numpy. Both gates run across all 7 corpus domains.

How the TOST gate works

Pre-register the indifference margin and significance level.
For each turn, run the agent on the original context and on the compressed context; record whether the decision matched.
Compute paired differences (compressed score − baseline score) across all turns.
Run the lower one-sided test. Reject H₀ at α → the strategy is certified non-inferior.

$ distil certify --strategy distil
decision-equivalence match rate: 100.0%
TOST non-inferiority (margin=0.02, alpha=0.05): mean diff=+0.000, p=0.0000

VERDICT: PASS  (certified non-inferior)

$ distil certify --strategy aggressive
decision-equivalence match rate: 0.0%
TOST non-inferiority (margin=0.02, alpha=0.05): mean diff=−1.000, p=1.0000

VERDICT: FAIL  (NOT certified — would degrade quality)

A strategy that cannot pass the gate simply does not ship. You can watch the gate reject a quality-degrading strategy yourself — it is not a configuration option.

3. Cache misses — not size — dominate agent cost

This is the insight most context compressors miss entirely, and it is where Distil's biggest wins come from.

In a multi-turn agent loop, the growing context is re-sent to the model at every turn. With prompt caching enabled:

A cache read costs roughly 0.1× the normal input price.
A cache write costs roughly 1.25× of normal input.
A cache miss (fresh token) costs the full input price.

The dominant cost lever is therefore not how many tokens you send — it is how many of those tokens hit the cache. A naive compressor that rewrites the context every turn drops the longest common prefix to zero, losing the 10× discount on every previously-cached block.

Naive recompression sends fewer tokens yet costs more than no compression at all, because it rewrites the cached prefix every turn and pays full input price on tokens that would otherwise cost 0.1×.

Distil's cache-aware engine models this explicitly. It keeps the prefix byte-stable across turns via schema canonicalization and volatile-field extraction, compresses only the volatile tail, and proves via simulation that the naive approach is strictly dominated.

The numbers, measured

33%

Cost cut, cache-aware lossless · ops/sre trajectory

−11%

Naive compress + cache (costs more than baseline)

10×

Cache-read discount vs. fresh input (Anthropic pricing)

Cache-aware savings vs naive compression

The 33% figure is the conservative lossless, single-trajectory result. Across a varied 64-trajectory corpus, the certified lossless ceiling is ~58%. The distil-causal mode reaches ~81% on that offline corpus. And it holds up where it counts — under live model grading: in the live head-to-head against the real installed packages (a realistic, decision-determined corpus, graded by claude-opus-4-8), Distil certified 83.2% savings at a 0% decision-change rate, while LLMLingua-2 reached only 53.1% savings and flipped 1-in-5 decisions, and Headroom achieved 35.3% at 0%.

For the compression mechanics, see Techniques. To price any trajectory yourself, see CLI Reference → distil savings.