Integrations — one proxy, every SDK

Start distil proxy once and point any SDK's baseURL at it. No library changes, no monkey-patching — compression happens at the network layer.

How it works

The proxy (distil proxy, default http://127.0.0.1:8788) is a local HTTP server. It intercepts the three compressible paths across all major LLM APIs:

• /v1/messages — Anthropic Messages API
• /v1/chat/completions — OpenAI Chat Completions (also used by LiteLLM, LangChain, etc.)
• /v1/responses — OpenAI Responses API
• /v1beta/models/{model}:generateContent and :streamGenerateContent — Google Gemini REST API (also the /v1/… host variants)

All other paths and HTTP verbs pass through unchanged. Your API key travels in the request headers exactly as normal — the proxy never logs or stores it.

SDK integration matrix

Snippets

Anthropic Python SDK

import anthropic

client = anthropic.Anthropic(
    api_key="sk-ant-…",
    base_url="http://127.0.0.1:8788",
)
response = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
)

OpenAI Python SDK

import openai

client = openai.OpenAI(
    api_key="sk-…",
    base_url="http://127.0.0.1:8788/v1",
)
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)

LiteLLM

import litellm

response = litellm.completion(
    model="claude-opus-4-5",
    api_base="http://127.0.0.1:8788",
    api_key="sk-ant-…",
    messages=[{"role": "user", "content": "Hello!"}],
)

Running the standalone LiteLLM Proxy instead?

Same mechanism, one YAML field: point each model's api_base at distil proxy and every request routed through the LiteLLM Proxy is compressed transparently, with no change to callers.

# Terminal 1 — distil sits in front of the real upstream
distil proxy --port 8788 --upstream https://api.anthropic.com

# config.yaml — Terminal 2
model_list:
  - model_name: claude-opus-4-5
    litellm_params:
      model: anthropic/claude-opus-4-5
      api_base: http://127.0.0.1:8788
      api_key: os.environ/ANTHROPIC_API_KEY

# Terminal 2 — start the LiteLLM Proxy against that config
litellm --config config.yaml

Vercel AI SDK (TypeScript)

import { createAnthropic } from "@ai-sdk/anthropic";
import { generateText } from "ai";

const anthropic = createAnthropic({
  baseURL: "http://127.0.0.1:8788",
  apiKey: process.env.ANTHROPIC_API_KEY,
});

const { text } = await generateText({
  model: anthropic("claude-opus-4-5"),
  prompt: "Hello!",
});

LangChain.js (TypeScript)

import { ChatAnthropic } from "@langchain/anthropic";

const model = new ChatAnthropic({
  model: "claude-opus-4-5",
  apiKey: process.env.ANTHROPIC_API_KEY,
  anthropicApiUrl: "http://127.0.0.1:8788",
  // older versions: clientOptions: { baseURL: "http://127.0.0.1:8788" }
});

const response = await model.invoke([
  ["human", "Hello!"],
]);

Google Gemini REST

# Start proxy pointing at the Gemini API
distil proxy --port 8788 --upstream https://generativelanguage.googleapis.com

import google.generativeai as genai

genai.configure(
    transport="rest",
    client_options={"api_endpoint": "http://127.0.0.1:8788"},
)
model = genai.GenerativeModel("gemini-1.5-pro")
response = model.generate_content("Hello!")

See examples/python_gemini.py for a full runnable example including tool-use turns. The proxy transparently compresses text parts (Tier-0 lossless) and functionResponse payloads (Tier-1 reversible digest). functionCall, inlineData, fileData, model-authored text, and systemInstruction are always passed through unchanged.

Claude Code plugin

The first-class integration: a session-first savings status line plus slash commands, shipped as a marketplace-format plugin (plugins/distil). Wire it in one step with distil setup (or distil onboard), then route the agent through compression with distil wrap -- claude.

The status line leads with this session (▼75.0K −62% $0.31 · Σ27.0M) and shows decision-equivalence only once 25+ shadow samples exist — a rate over a handful of samples is noise, and distil doesn't print noise.

MCP server

A zero-dependency Model Context Protocol server (stdlib only — no SDK) exposes distil's reversible compression to any MCP client (Claude Desktop, IDEs, agents) over stdio:

distil mcp        # newline-delimited JSON-RPC 2.0 over stdio

Tools: distil_compress(text) → a compact digest + an 8-hex handle (the original is kept in a local store, never returned until asked); distil_expand(handle) → the original text; distil_savings() → cumulative ledger savings. Wire it into an MCP client's server config as a stdio command (command: "distil", args: ["mcp"]).

In-process hooks (LiteLLM · LangChain · LangGraph)

Prefer not to run a sidecar? Compress the request in-process — the same reversible compression, no proxy. Every helper lazy-imports (or duck-types) its framework, so distil stays zero-dep.

# LiteLLM — drop-in for litellm.completion
from distil.integrations import litellm as distil_litellm
resp = distil_litellm.completion(model="claude-opus-4-8", messages=[...],
                                 distil_verbatim=True)  # optional, Tier-0 only

# LangChain — compress a message list before the model call (duck-typed)
from distil.integrations.langchain import compress_messages
msgs = compress_messages(state["messages"], verbatim=True)

# LangGraph — compress graph state right before the model node
from distil.integrations.langgraph import pre_model_hook
agent = create_react_agent(model, tools, pre_model_hook=pre_model_hook())

Tool/function messages get the reversible Tier-1 digest; human/system messages get Tier-0 lossless; the model's own words are never rewritten. compress() (LiteLLM), compress_messages() (LangChain), and pre_model_hook() (LangGraph) are framework-free and unit-tested. The LangGraph hook returns only the updated message list, so every other state field is left intact.

Observability headers

The proxy adds up to 9 response headers per compressed response. The first two appear on every compressed request; the rest are conditional:

The managed gateway (distil gateway) additionally adds x-distil-tenant: <id> for per-tenant accounting.

Node.js / TypeScript

Distil's universal path is the proxy — it's language-agnostic, so JS/TS stacks need no Distil-specific package. Start the proxy (or wrap your agent) and point your SDK's baseURL at it:

# start the proxy (needs Python 3.11+ — see Install)
distil proxy --port 8788 --upstream https://api.anthropic.com

// then in your Node/TS app — no code change beyond baseURL
const client = new Anthropic({ baseURL: "http://localhost:8788" });

Or wrap an existing CLI/agent in one shot: distil wrap -- <your-command> starts the proxy and injects ANTHROPIC_BASE_URL automatically.

Homebrew

brew tap dshakes/tap
brew install dshakes/tap/distil
distil proxy --port 8788