Metadata-Version: 2.4
Name: liquid-memory-proxy
Version: 2.3.1
Summary: Fact-preserving prompt compression as a drop-in proxy for OpenAI, Anthropic, and Google.
Author-email: Jamie Obala <jamieobala2028@u.northwestern.edu>, Selina Sun <selinasun2028@u.northwestern.edu>
License: Proprietary
Project-URL: Homepage, https://liquid-memory.vercel.app
Project-URL: Repository, https://github.com/Jamie2111/liquid_memory
Project-URL: Issues, https://github.com/Jamie2111/liquid_memory/issues
Project-URL: Documentation, https://github.com/Jamie2111/liquid_memory#readme
Keywords: llm,compression,openai,proxy,kv-cache,long-context,inference,cost-optimization
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Internet :: Proxy Servers
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: fastapi>=0.110
Requires-Dist: uvicorn[standard]>=0.30
Requires-Dist: litellm>=1.51
Requires-Dist: pydantic>=2.5
Requires-Dist: python-dotenv>=1.0
Requires-Dist: requests>=2.31
Provides-Extra: mode2
Requires-Dist: torch>=2.3; extra == "mode2"
Requires-Dist: transformers<5,>=4.40; extra == "mode2"
Provides-Extra: vllm
Requires-Dist: vllm<0.11,>=0.10.2; extra == "vllm"
Requires-Dist: hf_transfer; extra == "vllm"
Provides-Extra: cache
Requires-Dist: redis>=5.0; extra == "cache"
Provides-Extra: fidelity
Requires-Dist: datasets>=2.18; extra == "fidelity"
Requires-Dist: tiktoken>=0.7; extra == "fidelity"
Provides-Extra: swebench
Requires-Dist: swebench>=2.1; extra == "swebench"
Requires-Dist: datasets>=2.18; extra == "swebench"
Requires-Dist: tiktoken>=0.7; extra == "swebench"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: twine>=5.1; extra == "dev"

# Liquid Memory

**A drop-in proxy that compresses an AI agent's context faithfully, before it reaches the model.**

[![PyPI](https://img.shields.io/pypi/v/liquid-memory-proxy)](https://pypi.org/project/liquid-memory-proxy/)
[![CI](https://github.com/Jamie2111/liquid_memory/actions/workflows/install.yml/badge.svg)](https://github.com/Jamie2111/liquid_memory/actions)

Website: [liquid-memory.vercel.app](https://liquid-memory.vercel.app)

---

Coding agents burn tokens on context that piles up across turns: stale file re-reads,
old tool output, superseded views. Liquid Memory sits in the request path, removes that
redundant prior-turn context, and forwards the request to your real provider. It only ever
shrinks the **input**; it never touches the model's **output**. Everything it keeps is
byte-exact. It never paraphrases.

## Two guarantees, by construction

**1. It never silently alters your inputs.** Compression is extractive: it selects which
source spans to keep and copies them byte-for-byte. A file path, an identifier, a line
number, a UUID, or a numeric constant can never be corrupted. What survives is identical to
what you sent. When a load-bearing span cannot be kept safely, the coverage gate forwards
your full prompt unchanged. It **fails open**: a bug in the proxy can never break your run,
the worst case is no compression this turn.

**2. It never costs more than calling the model directly.** A savings floor sits in front of
every request. If compressing would not remove enough tokens, the request passes through
byte-identical to going direct. Your worst case is exactly the bill you have today.

## Install

```bash
pip install liquid-memory-proxy
export LM_COMPRESSION=balanced        # lossless | balanced | aggressive | max
liquid-memory start                   # serves http://localhost:8000
```

Verify it is up: `curl http://localhost:8000/healthz`

(Optional: `export LM_AUTH_KEY="lm_live_..."` from the portal at
[liquid-memory.vercel.app](https://liquid-memory.vercel.app) attributes usage to your
account. Omit it, or set `LM_SKIP_AUTH=1`, to evaluate without a key.)

Then point your existing client at the proxy. Your API key does not change; Liquid Memory
forwards it to the provider untouched.

| Client | Change | Liquid Memory serves |
|--------|--------|----------------------|
| OpenAI SDK / chat models | `base_url` -> `http://localhost:8000/v1` | `/v1/chat/completions` |
| OpenAI Responses (gpt-5.x / codex) | same base URL | `/v1/responses` |
| Claude Code / Claude Agent SDK / Anthropic SDK | `ANTHROPIC_BASE_URL` -> `http://localhost:8000` | native `/v1/messages` |
| Google Gemini SDK | client base -> `http://localhost:8000` | `generateContent` / `streamGenerateContent` |

That is the whole integration. No new SDK, no restructured messages, no code change beyond
the base URL.

## Look before you spend

Add `?lm_preview=1` to any request and Liquid Memory returns the exact compression **plan**,
tokens removed, what stays byte-exact, whether the failure signal survives, and forwards
nothing. No token is spent. Verify it on your own messy agent trace before putting it on the
hot path.

```bash
curl -s "http://localhost:8000/v1/chat/completions?lm_preview=1" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.5","messages":[ ...your trajectory... ]}'
```

`POST /v1/context/report` does the same analysis offline for any context (protected vs
compressible tokens, projected removal, fidelity checks), with no model call. A coding agent
can fetch `GET /skill.md` to configure itself against the proxy.

## Does it hold up? (measured, not asserted)

**Resolve parity on SWE-bench Verified.** Agents running through Liquid Memory resolve the
same tasks as going direct: **60/98 with LM vs 57/99 direct** (gpt-5.3-codex), within noise.
Compression does not damage task success. The point is not a resolve gain, it is compression
at no measured cost to correctness.

**vs the naive alternative.** Cutting old context is easy; cutting the *right* context is not.
On a real coding trajectory:

| Approach | Input removed | Traceback kept | Target function kept | Live turn byte-exact |
|----------|---------------|----------------|----------------------|----------------------|
| naive truncation | 76% | **no** | yes | yes |
| Liquid Memory (max) | 65% | yes | yes | yes |

Naive truncation removes more tokens and silently drops the traceback the agent was
debugging. Liquid Memory removes almost as much and keeps every load-bearing span byte-exact.
Reproduce it: `python benchmarks/compare_baselines.py`.

## When it actually saves money (the honest version)

Liquid Memory compresses **input**. That has a real limit worth stating plainly:

| Your situation | Does it cut the bill? | Quality |
|----------------|-----------------------|---------|
| Caching un-optimized: no `cache_control` on Claude, bursty traffic, unstable prefixes | **Yes**, real savings | parity |
| Fully cache-optimized: warm caches, stable prefixes | No, break-even (the never-costs-more guard) | parity, plus more context headroom and throughput |

The deciding question for any workload is not "which model" but **"are you already capturing
the caching that's available?"** Most teams are not. Independent of caching, Liquid Memory
always gives: resolve parity, context-window headroom on long trajectories, and more work
under a TPM rate limit (fewer tokens per minute).

## The dial

`LM_COMPRESSION` sets how much prior-turn context is removed by controlling the recent
verbatim window:

- `lossless` strict gate, only provably-safe removals (exact re-reads, duplicates)
- `balanced` default, wide recent verbatim window
- `aggressive` smaller window, more removed
- `max` smallest window, use when context-window pressure is the problem

The live turn, every file's current state, and any failure signal are kept byte-exact at
every dial. Start at `balanced`, measure with preview, dial up as your workload tolerates.

## Trust layer (all opt-in, off by default)

Because Liquid Memory sits in the request path, it can optionally provide:

- **Hash-chained audit log** (`LM_AUDIT=1`): a tamper-evident, append-only record of every
  turn (model call, messages, output, tool calls). Verify any time with
  `liquid-memory audit-verify`; key it with `LM_AUDIT_KEY` held off the audited box to make
  forged entries impossible.
- **Guardrails**: block destructive agent actions in-path.
- **Loop guard**: detect a provably-stuck run before it burns its budget.
- **Tool-call repair** (`LM_REPAIR_TOOLCALLS=1`): fix malformed tool calls in flight using
  the request's own schema. Conservative, never touches a clean call, never fabricates.

Every response also carries `X-LiquidMemory-Overhead-Ms` (the proxy's own measured latency)
and `X-LiquidMemory-Degraded` (whether it failed open). `GET /stats` reports p50/p95/max
overhead and measured savings on your own traffic. Nothing is stored server-side by default
and nothing phones home.

## Pricing

Commission on **measured** savings only. No savings, no charge. `/stats` reports the savings
figure the commission is computed from.

## Self-hosted and private

The proxy is one stateless process (`liquid-memory start`, or the Docker image) with a
`GET /healthz` liveness probe. It runs on your own machine or VPC; context only ever goes to
the provider you already call. Liquid Memory adds no external hop and stores no message
content by default.

## Contact

jamieobala2028@u.northwestern.edu, selinasun2028@u.northwestern.edu
For security issues: same addresses, acknowledged within one business day.

License: Proprietary. Copyright 2026 Liquid Memory, Inc.

<sub>A separate linear-attention library for teams training or self-hosting their own model
(a drop-in `torch.nn.MultiheadAttention` replacement with linear-time scaling) is available
under separate license. See [`README_INTEGRATION.md`](README_INTEGRATION.md) or email the
founders.</sub>
