Integrations — one proxy, every SDK

Start distil proxy once and point any SDK's baseURL at it. No library changes, no monkey-patching — compression happens at the network layer.

How it works

The proxy (distil proxy, default http://127.0.0.1:8788) is a local HTTP server. It intercepts the three compressible paths across all major LLM APIs:

• /v1/messages — Anthropic Messages API
• /v1/chat/completions — OpenAI Chat Completions (also used by LiteLLM, LangChain, etc.)
• /v1/responses — OpenAI Responses API
• /v1beta/models/{model}:generateContent and :streamGenerateContent — Google Gemini REST API (also the /v1/… host variants)

All other paths and HTTP verbs pass through unchanged. Your API key travels in the request headers exactly as normal — the proxy never logs or stores it.

SDK integration matrix

Snippets

Anthropic Python SDK

import anthropic

client = anthropic.Anthropic(
    api_key="sk-ant-…",
    base_url="http://127.0.0.1:8788",
)
response = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
)

OpenAI Python SDK

import openai

client = openai.OpenAI(
    api_key="sk-…",
    base_url="http://127.0.0.1:8788/v1",
)
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)

LiteLLM

import litellm

response = litellm.completion(
    model="claude-opus-4-5",
    api_base="http://127.0.0.1:8788",
    api_key="sk-ant-…",
    messages=[{"role": "user", "content": "Hello!"}],
)

Running the standalone LiteLLM Proxy instead?

Same mechanism, one YAML field: point each model's api_base at distil proxy, and every request routed through the LiteLLM Proxy is compressed transparently.

# Terminal 1 — distil sits in front of the real upstream
distil proxy --port 8788 --upstream https://api.anthropic.com

# config.yaml — Terminal 2
model_list:
  - model_name: claude-opus-4-5
    litellm_params:
      model: anthropic/claude-opus-4-5
      api_base: http://127.0.0.1:8788
      api_key: os.environ/ANTHROPIC_API_KEY

# Terminal 2 — start the LiteLLM Proxy against that config
litellm --config config.yaml

Vercel AI SDK (TypeScript)

import { createAnthropic } from "@ai-sdk/anthropic";
import { generateText } from "ai";

const anthropic = createAnthropic({
  baseURL: "http://127.0.0.1:8788",
  apiKey: process.env.ANTHROPIC_API_KEY,
});

const { text } = await generateText({
  model: anthropic("claude-opus-4-5"),
  prompt: "Hello!",
});

LangChain.js (TypeScript)

import { ChatAnthropic } from "@langchain/anthropic";

const model = new ChatAnthropic({
  model: "claude-opus-4-5",
  apiKey: process.env.ANTHROPIC_API_KEY,
  anthropicApiUrl: "http://127.0.0.1:8788",
  // older versions: clientOptions: { baseURL: "http://127.0.0.1:8788" }
});

const response = await model.invoke([
  ["human", "Hello!"],
]);

Google Gemini REST

# Start proxy pointing at the Gemini API
distil proxy --port 8788 --upstream https://generativelanguage.googleapis.com

import google.generativeai as genai

genai.configure(
    transport="rest",
    client_options={"api_endpoint": "http://127.0.0.1:8788"},
)
model = genai.GenerativeModel("gemini-1.5-pro")
response = model.generate_content("Hello!")

See examples/python_gemini.py for a full runnable example including tool-use turns. The proxy transparently compresses text parts (Tier-0 lossless) and functionResponse payloads (Tier-1 reversible digest). functionCall, inlineData, fileData, model-authored text, and systemInstruction are always passed through unchanged.

Claude Code plugin

The first-class integration: a session-first savings status line plus slash commands, shipped as a marketplace-format plugin (plugins/distil). Wire it in one step with distil setup (or distil onboard), then route the agent through compression with distil wrap -- claude.

One pattern in every state: distil · <live> · total ▼<lifetime>. The live segment aggregates recent activity (last 15 min, all terminals), so it never flickers between sessions:

▼ = tokens saved · total = lifetime · ✓eq = decision-equivalence (shown only past 25 shadow samples — a rate over a handful is noise). Sharing the line with git/cwd/model? Set DISTIL_STATUSLINE=minimal for a two-fact segment: distil ▼75.0K · 27.0M total.

MCP server

A zero-dependency Model Context Protocol server (stdlib only — no SDK) exposes distil's reversible compression to any MCP client (Claude Desktop, IDEs, agents) over stdio:

claude mcp add distil -- distil mcp   # Claude Code — one line, done

Every other client takes the same stdio config — Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json), Cursor (.cursor/mcp.json), VS Code (.vscode/mcp.json):

{
  "mcpServers": {
    "distil": { "command": "distil", "args": ["mcp"] }
  }
}

No distil install? Run it from PyPI on demand — "command": "uvx", "args": ["--from", "distil-llm", "distil", "mcp"]. Restart the client after editing. To check the server without a client at all:

echo '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | distil mcp

Each tool carries MCP annotations (readOnlyHint, idempotentHint, openWorldHint: false), so a client knows distil_expand is a safe, repeatable, offline read instead of inferring it from prose.

This is the recall path, not the savings path. The MCP server does not compress your agent's traffic — distil wrap does that transparently, with no tool calls. What the MCP server adds is the other half: any agent, including one you never wrapped, can call distil_expand on a handle it sees in context and get the original back. Handles survive restarts and cross processes, and age out after DISTIL_RESTORE_TTL_DAYS (default 14).

In-process hooks (LiteLLM · LangChain · LangGraph)

Prefer not to run a sidecar? Compress the request in-process — the same reversible compression, no proxy. Every helper lazy-imports (or duck-types) its framework, so distil stays zero-dep.

# LiteLLM — drop-in for litellm.completion
from distil.integrations import litellm as distil_litellm
resp = distil_litellm.completion(model="claude-opus-4-8", messages=[...],
                                 distil_verbatim=True)  # optional, Tier-0 only

# LangChain — compress a message list before the model call (duck-typed)
from distil.integrations.langchain import compress_messages
msgs = compress_messages(state["messages"], verbatim=True)

# LangGraph — compress graph state right before the model node
from distil.integrations.langgraph import pre_model_hook
agent = create_react_agent(model, tools, pre_model_hook=pre_model_hook())

Tool/function messages get the reversible Tier-1 digest; human/system messages get Tier-0 lossless; the model's own words are never rewritten. compress() (LiteLLM), compress_messages() (LangChain), and pre_model_hook() (LangGraph) are framework-free and unit-tested. The LangGraph hook returns only the updated message list, so every other state field is left intact.

Observability headers

The proxy adds up to 9 response headers per compressed response. The first two appear on every compressed request; the rest are conditional:

The managed gateway (distil gateway) additionally adds x-distil-tenant: <id> for per-tenant accounting.

Node.js / TypeScript

Distil's universal path is the proxy — it's language-agnostic, so JS/TS stacks need no Distil-specific package. Start the proxy (or wrap your agent) and point your SDK's baseURL at it:

# start the proxy (needs Python 3.11+ — see Install)
distil proxy --port 8788 --upstream https://api.anthropic.com

// then in your Node/TS app — no code change beyond baseURL
const client = new Anthropic({ baseURL: "http://localhost:8788" });

Or wrap an existing CLI/agent in one shot: distil wrap -- <your-command> starts the proxy and injects ANTHROPIC_BASE_URL automatically.

Homebrew

brew tap dshakes/tap
brew install dshakes/tap/distil
distil proxy --port 8788