Core Concepts

Three ideas underpin everything Distil does. Understanding them explains why the numbers are real and why the approach is different.

1. Decision-equivalence, not byte-equivalence

Most context compressors frame the problem as a string transformation: compress the context, compare the result byte-by-byte to the original, report the overlap as accuracy. That framing has a fatal flaw — byte-equivalence and high compression are information-theoretically in tension. You cannot meaningfully claim "87% compression, 100% accuracy" under byte-equivalence; the two numbers cannot describe the same operation.

Decision-equivalence is the correct target. An agent only needs to take the same actions and produce the same outputs whether or not its context was compressed. "Same decision" is:

• The same tool calls (name + arguments) in the same order.
• The same final answer or action at each turn.
• Measured across a real multi-turn trajectory, not a single prompt.

This is measurable. It is certifiable. And it is compatible with aggressive compression — because context blocks that are causally inert (never change a decision when removed) are provably free to drop, regardless of how many bytes they contain.

2. The quality contract

A compression strategy in Distil does not ship unless a pre-registered TOST non-inferiority test certifies it. TOST (Two One-Sided Tests) is the standard statistical framework for equivalence testing — the same method used in pharmaceutical non-inferiority trials.

How the gate works

1. Pre-register an indifference margin (default: 0.02 — tolerate at most a 2-point drop in task-success rate) and a significance level (default: α = 0.05).
2. For each turn in the trajectory, run the agent on the original context (baseline) and on the compressed context, and record whether the decision matched.
3. Compute paired differences (compressed score − baseline score) across all turns.
4. Run the lower one-sided test: reject H₀ that mean ≤ −margin. If you can reject that hypothesis at α, the strategy is certified non-inferior.

The Student-t tail is computed from a hand-rolled regularised incomplete beta function — zero dependencies on scipy or numpy. The contract is the same gate, run across all 7 corpus domains.

$ distil certify --strategy distil
decision-equivalence match rate: 100.0%
TOST non-inferiority (margin=0.02, alpha=0.05): mean diff=+0.000, p=0.0000

VERDICT: PASS  (certified non-inferior)

$ distil certify --strategy aggressive
decision-equivalence match rate: 0.0%
TOST non-inferiority (margin=0.02, alpha=0.05): mean diff=−1.000, p=1.0000

VERDICT: FAIL  (NOT certified — would degrade quality)

A strategy that cannot pass the gate simply does not ship. You can watch the gate reject a quality-degrading strategy yourself — it is not a configuration option.

3. Cache misses — not size — dominate agent cost

This is the insight most context compressors miss entirely, and it is where Distil's biggest wins come from.

In a multi-turn agent loop, the growing context is re-sent to the model at every turn. With prompt caching enabled:

• A cache read costs roughly 0.1× (one tenth) of the normal input price.
• A cache write costs roughly 1.25× of normal input.
• A cache miss (fresh token) costs the full input price.

The dominant cost lever is therefore not how many tokens you send — it is how many of those tokens hit the cache. A naive compressor that rewrites the context every turn causes the longest common prefix to drop to zero, losing the 10× discount on every cached block. The result:

Distil's cache-aware engine models this explicitly. It keeps the prefix byte-stable across turns (via schema canonicalization and volatile-field extraction), compresses only the volatile tail, and proves via simulation that the naive approach is strictly dominated.

The numbers, measured

See Techniques for the full breakdown of how cache stabilization works, and CLI Reference for distil savings which prices all four strategies on any trajectory.