Compression Techniques

Every technique in Distil targets a specific slice of agent cost. Two carry the bulk of the win; the rest stack cleanly on top.

① Cache-aware compression

Module: distil/compress/cache_aware.py · Loss profile: lossless

The dominant cost lever in a multi-turn agent loop is not token count — it is cache hit rate. A cache read costs ~0.1× fresh input; a cache miss costs the full price. Distil models the full multi-turn loop, tracking the longest contiguous prefix that is identical between consecutive turns. Any compressor that rewrites the prefix (even to save tokens) destroys this prefix match and loses the 10× discount.

Cache stabilization

To keep the prefix byte-stable turn over turn, Distil applies two lossless pre-processing steps before any compression runs:

• Schema canonicalization — recursively sorts JSON object keys in tool payloads and API responses. Two semantically identical JSON objects with different key orders produce byte-identical text after canonicalization, which means they hash identically and hit the prefix cache.
• Volatile-field extraction — fields whose values change every turn (timestamps, UUIDs, JWT tokens, request IDs) are lifted out of the stable prefix into the volatile tail. The stable portion stops churning; the volatile tail is compressed separately.

The simulation

cache_aware.simulate() models four strategies across a full trajectory and reports per-turn cost breakdown (cache reads, cache writes, fresh tokens). Run distil savings to see the numbers on any trajectory.

$ distil savings --pricing claude-opus-4-8
strategy                               $ / run   vs baseline  cache hits
------------------------------------------------------------------------
baseline (no cache, no compress)       0.01524          0.0%           0
cache only                             0.01115         26.8%       1,028
naive compress + cache                 0.01691        -11.0%           0  ← busts cache
distil (cache-aware lossless)          0.01019         33.1%       1,028

④ Causal / counterfactual pruning

Module: distil/replay/ablation.py · Loss profile: certified

The eval engine in Distil is not a ruler bolted on the side of compression — it is a discovery engine. The algorithm:

1. For each context block present in the trajectory, remove it and replay the trajectory.
2. Record whether any turn's decision changed.
3. A block that never changed a decision in any of its occurrences is causally inert — provably free to drop.

This is the sense in which the measurement produces the compression policy. Speculative retrievals that were never cited, stale history that no longer affects any tool call, diagnostic context that was superseded — these all show up as zero-impact in ablation and can be pruned with a causal guarantee, not a heuristic guess.

$ distil prune
causal ablation over 'sre-disk-incident' — what is free to drop?

block                  occ   tokens  verdict
doc-0                    4      312  PRUNE (causally inert)
doc-1                    4      303  PRUNE (causally inert)
obs-0                    4      234  keep (changed a decision)
obs-1                    4      198  keep (changed a decision)
system                   4      218  keep (changed a decision)

tokens provably free to drop: 615 across 2 block(s).

Risk-graded tiers

Every technique is assigned to a tier based on its loss profile. Distil applies them in order — the safest ones unconditionally, the certified ones only when the gate passes.

Learned keep-model codec

Module: distil/codec/learned.py · Loss profile: pluggable

For per-content-type decisions about which lines of a tool result are worth keeping, Distil ships a logistic-regression keep classifier — trained from scratch in pure Python, zero external dependencies.

• Architecture: logistic regression over a fixed feature set (codec/features.py), trained with full-batch gradient descent on binary cross-entropy with L2 regularization.
• Training signal: labels are distilled from SalienceKeepModel — a line is labelled keep (1.0) if the heuristic salience score is ≥ 0.6.
• Performance on held-out lines: 96.4% accuracy, 0.98 F1, deterministic train/test split (SHA-256 hash-based, reproducible).
• Persistence: weights stored in distil/codec/weights.json; load with LogisticKeepModel.load().

Gist caching

Module: distil/gist.py · Loss profile: lossless

Tool schemas are verbose and essentially never change within a session. Gist caching detects when the same tool schema has been sent before (by content hash) and replaces subsequent occurrences with a compact reference handle. The model only needs to see the full schema once; subsequent turns pay only for the reference token.

This is implemented as a content-addressed local cache — no network calls, no external state. The handle is embedded inline in the message; a session-aware proxy or adapter layer resolves it back to the full schema if needed.

Delta / append-only context

Module: distil/delta.py · Loss profile: lossless

Context history is append-only — a block present in turn N must carry identical bytes in turn N+1 (this is enforced by the byte-fidelity gate). The delta engine exploits this: only the new blocks added since the last turn need to be transmitted at full cost. Stable history is already in the cache and costs only cache-read price. The delta layer ensures no block is re-sent in a way that would corrupt the prefix hash.

BM25 partial retrieval

Module: distil/retrieval.py

When a context block is digested (Tier 1) and represented by a handle, BM25 partial retrieval lets the agent query for the most relevant chunks of the original without re-expanding the entire block. This keeps the context window focused while preserving access to the full original content on demand — useful for large tool outputs, retrieved documents, or log dumps where only a fraction is decision-relevant.