Adapters & Integration

Two paths to production: a one-line Python wrapper that compresses in-process, or a standalone HTTP proxy that works with any SDK or framework.

Path 1 — In-process: wrap(client)

Module: distil/adapters/anthropic.py

The fastest path for Python codebases already using the Anthropic SDK. Wrap your client once at construction time — all downstream messages.create calls are transparently compressed and cache-pinned with no call-site changes.

import anthropic
from distil.adapters.anthropic import wrap

# Before: plain Anthropic client
client = anthropic.Anthropic()

# After: one-line drop-in
client = wrap(anthropic.Anthropic())

# No other code changes — all calls compressed transparently
response = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    system="You are a helpful assistant.",
    messages=[{"role": "user", "content": "Analyse this log output…"}],
)

What the wrapper does

On every messages.create(**kwargs) call, the wrapper:

1. Compresses the messages array via compress_messages():
   • User text blocks: Tier-0 lossless transforms (JSON minify + run collapse).
   • Tool result blocks with ≥ 6 lines: Tier-1 reversible digest, original stored in a local RestoreStore.
   • Assistant text, tool_use, and image blocks: passed through unchanged.
2. Pins the cache prefix via place_cache_control(): marks the last stable system block with {"cache_control": {"type": "ephemeral"}}, so the first call pays the cache-write price and every subsequent call pays the ~0.1× cache-read price.
3. Forwards the modified kwargs to the real client.messages.create.

The RestoreStore

Digested blocks embed an 8-hex SHA-256 handle in their compressed text. The RestoreStore maps handles to originals locally — it is never sent to the model and costs zero tokens. To recover an original:

from distil.adapters.anthropic import compress_messages

compressed, store = compress_messages(messages)
# store.handles → frozenset of all active handles
original = store.expand("a3f92b1c")  # retrieve original by handle

Design properties

• Duck-typed. The wrapper imports nothing from the Anthropic SDK — it works with any object that exposes a messages.create(**kwargs) method, including mocks in test environments.
• Non-mutating. Input message lists are never modified in place; new lists are returned.
• Reversible. Every change is either lossless (Tier-0) or stored in the local RestoreStore (Tier-1). Nothing is permanently lost.

Path 2 — Provider proxy: distil proxy

Module: distil/proxy.py

A lightweight HTTP proxy that sits between your client and the real LLM API. It intercepts POST /v1/messages, POST /v1/chat/completions, and POST /v1/responses requests, compresses the messages array, and forwards the modified request to the upstream. All other paths and methods pass through unchanged.

This approach works with any SDK, framework, or language — you do not need to touch your application code at all.

$ python -m distil.proxy --port 8788 --upstream https://api.anthropic.com
distil proxy listening on http://127.0.0.1:8788
  → upstream: https://api.anthropic.com

Point your client at the proxy

import anthropic
client = anthropic.Anthropic(
    base_url="http://localhost:8788"
)

import openai
client = openai.OpenAI(
    base_url="http://localhost:8788/v1",
    api_key="…"
)

litellm.completion(
    api_base="http://localhost:8788",
    model="anthropic/claude-…",
    messages=[...]
)

from langchain_anthropic import ChatAnthropic
llm = ChatAnthropic(
    base_url="http://localhost:8788"
)

Proxy flags

Response headers

The proxy adds two custom headers to compressed responses:

• x-distil-compressed: 1 — signals that the request was compressed
• x-distil-tokens-saved: <n> — heuristic estimate of tokens saved (not billing-grade)

Threading model

The proxy uses Python's ThreadingHTTPServer — each incoming connection is handled in its own thread, so concurrent requests from multiple clients are supported. It is intentionally simple and easy to audit.

Choosing between in-process and proxy

Honest note: live provider path

The --runner anthropic flag on distil certify also routes to the live Anthropic API. It is implemented and wired up, but marked UNVERIFIED in the README until you run it with a real key. The offline deterministic runner (the default) is what powers all corpus measurements.