Metadata-Version: 2.4
Name: tokensqueeze
Version: 0.2.0
Summary: Reduce LLM conversation tokens by up to 70%
Project-URL: Homepage, https://github.com/StuckInTheNet/tokensqueeze
Project-URL: Repository, https://github.com/StuckInTheNet/tokensqueeze
Project-URL: Issues, https://github.com/StuckInTheNet/tokensqueeze/issues
Author: Matt Fisher
License-Expression: MIT
License-File: LICENSE
Keywords: compression,cost-reduction,llm,openai,tokens
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.10
Requires-Dist: tiktoken>=0.7.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4.0; extra == 'dev'
Description-Content-Type: text/markdown

```
  _        _                                             
 | |_ ___ | | _____ _ __  ___  __ _ _   _  ___  ___ _______ 
 | __/ _ \| |/ / _ \ '_ \/ __|/ _` | | | |/ _ \/ _ \_  / _ \
 | || (_) |   <  __/ | | \__ \ (_| | |_| |  __/  __// /  __/
  \__\___/|_|\_\___|_| |_|___/\__, |\__,_|\___|\___/___\___|
                                 |_|                         
```

**Reduce LLM conversation tokens by up to 91%.** Drop-in compression for any OpenAI-compatible API.

[![CI](https://github.com/StuckInTheNet/tokensqueeze/actions/workflows/ci.yml/badge.svg)](https://github.com/StuckInTheNet/tokensqueeze/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)

---

## Why

Every turn in an LLM conversation resends the **entire history**. A 30-turn conversation can burn 50,000+ tokens per request — most of it redundant context the model has already processed.

**That's wasted money and wasted context window.**

tokensqueeze compresses your conversation history before each API call. One line of code, 91% fewer tokens.

## Real-World Results

Tested on **20 real Claude Code conversations** (1,200 messages):

```
  114,481 total tokens → 10,146 tokens (91.1% savings)
```

| Conversation | Messages | Before | After | Saved |
|---|---|---|---|---|
| Coding session #1 | 60 | 7,686 | 261 | **96.6%** |
| Coding session #2 | 60 | 11,356 | 377 | **96.7%** |
| Coding session #3 | 60 | 6,524 | 246 | **96.2%** |
| *Average (20 conversations)* | *60* | *5,724* | *507* | ***91.1%*** |

These aren't synthetic benchmarks — they're actual development conversations with tool calls, file reads, and multi-step debugging.

## Install

```bash
pip install tokensqueeze
```

For LLM-powered compression (recommended), install [Ollama](https://ollama.com) and pull a small model:

```bash
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2:3b   # 2GB, runs on any machine
```

## Quick Start

### Use with Claude Code, Cursor, or any AI tool

tokensqueeze runs as a local proxy — no code changes needed:

```bash
# Terminal 1: start the proxy
tokensqueeze proxy

# Terminal 2: use Claude Code through it
ANTHROPIC_BASE_URL=http://localhost:8080 claude
```

That's it. Every API call now gets compressed automatically. The proxy logs savings in real time:

```
[tokensqueeze] /v1/messages — 12,847 → 3,854 tokens (70% saved)
[tokensqueeze] /v1/messages — 18,231 → 5,102 tokens (72% saved)
```

Works with anything that lets you set a base URL: Claude Code, Cursor, OpenAI SDK apps, LangChain, etc.

For maximum compression (requires [Ollama](https://ollama.com)):
```bash
tokensqueeze proxy --use-llm
```

### Use as a Python library

If you're building your own app, one line:

```python
from openai import OpenAI
from tokensqueeze import squeeze

client = OpenAI()
messages = [...]  # your conversation history

result = squeeze(messages, use_llm=True)
print(result)  # CompressionResult(5,724 → 507 tokens, saved 91.1%)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=result.messages  # compressed
)
```

## How It Works

Four strategies applied in sequence, each targeting a different type of waste:

```
messages → [Dedup] → [Trim] → [Condense] → [Summarize] → compressed
```

| # | Strategy | What it does | Savings |
|---|----------|-------------|---------|
| 1 | **Deduplication** | Removes repeated content across messages (re-pasted code, duplicate tool results) | 5-15% |
| 2 | **Tool result trimming** | Shrinks large file reads and API responses that have already been processed | 15-25% |
| 3 | **Condensation** | Compresses individual verbose messages in place via local LLM | 20-40% |
| 4 | **LLM summarization** | Compresses old turns into structured fact-preserving notes | 30-50% |

Strategies are composable — use the defaults, pick specific ones, or write your own.

### What compressed context looks like

```
[Conversation context — 14 earlier messages]
- Topic: Troubleshooting Flask application issues
- ERROR: ImportError: cannot import name celery_app → Fix: separate celery_config.py
- ERROR: ValueError: invalid literal for int() → FIX: @app.route(/users/<int:id>)
- ISSUE: stale cache data → FIX: cache.delete(f"user:{user_id}") after commit
- ERROR: .exe upload accepted → FIX: extension allowlist + python-magic MIME check
- ISSUE: race condition on inventory → FIX: SELECT FOR UPDATE with_for_update()
```

Exact error messages, function names, and decisions are preserved — not just a vague summary.

## Two Modes

| | Extractive | LLM (Ollama) |
|---|---|---|
| **How** | Keyword extraction + sliding window | Local LLM condensation + structured summarization |
| **Speed** | <1ms | 1-15s |
| **Requires** | tiktoken only | Ollama + llama3.2:3b (free) |
| **Savings** | 40-70% | 85-97% |
| **Quality** | 81/100 | 79-89/100 |

```python
# Extractive — fast, no dependencies
result = squeeze(messages)

# LLM — much better compression, requires Ollama
result = squeeze(messages, use_llm=True)
```

If Ollama isn't running, `use_llm=True` falls back to extractive automatically.

## CLI

```bash
# Start the compression proxy
tokensqueeze proxy                              # Anthropic (default)
tokensqueeze proxy --target openai              # OpenAI
tokensqueeze proxy --use-llm                    # LLM compression via Ollama
tokensqueeze proxy --port 9000                  # Custom port

# Compress a conversation file
tokensqueeze compress conversation.json -o compressed.json
tokensqueeze compress --use-llm --stats-only conversation.json
tokensqueeze compress --use-llm --score conversation.json
```

## Quality Scoring

tokensqueeze includes a built-in quality judge that measures whether compressed conversations still produce equivalent LLM responses:

```python
result = squeeze(messages, use_llm=True)
quality = result.score_quality()

print(quality)         # QualityScore(overall=79/100, faith=80, comp=70, coher=90)
print(quality.grade)   # "good"
```

| Score | Grade | Meaning |
|---|---|---|
| 90-100 | Lossless | All meaningful context preserved |
| 70-89 | Good | Minor details lost, core meaning intact |
| 50-69 | Degraded | Noticeable information loss |
| <50 | Broken | Too much context destroyed |

## Advanced Usage

<details>
<summary>Custom strategy pipeline</summary>

```python
from tokensqueeze import squeeze, LLMSummarizeStrategy, ToolResultTrimStrategy, CondenseStrategy

result = squeeze(
    messages,
    strategies=[
        ToolResultTrimStrategy(max_tokens=100, preserve_recent=1),
        CondenseStrategy(min_tokens=50),
        LLMSummarizeStrategy(window_size=2, ollama_model="llama3.2:3b"),
    ]
)
```

</details>

<details>
<summary>Compressor instance with timeout</summary>

```python
from tokensqueeze import Compressor

compressor = Compressor(model="gpt-4o", max_time=30.0)  # 30s pipeline timeout
result = compressor.compress(messages)

print(f"Saved {result.saved_tokens:,} tokens ({result.savings_pct:.1f}%)")
for strategy, saved in result.strategy_stats.items():
    print(f"  {strategy}: -{saved:,}")
```

</details>

<details>
<summary>Custom strategy</summary>

```python
from tokensqueeze import Strategy

class MyStrategy(Strategy):
    def compress(self, messages, **kwargs):
        # Your compression logic here
        return messages
```

</details>

<details>
<summary>Remote or alternative LLM</summary>

```python
result = squeeze(
    messages,
    use_llm=True,
    ollama_model="mistral:7b",
    ollama_url="http://your-server:11434",
)
```

Any OpenAI-compatible endpoint works — Ollama, vLLM, LM Studio, or a remote server.

</details>

## Input Format

Standard OpenAI message format:

```json
[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "Hello"},
  {"role": "assistant", "content": "Hi there!"},
  {"role": "user", "content": "..."}
]
```

## How tokensqueeze Compares

Other tools in the token reduction space solve different parts of the problem:

| | **tokensqueeze** | **[Caveman](https://github.com/juliusbrussee/caveman)** | **[RTK](https://github.com/rtk-ai/rtk)** |
|---|---|---|---|
| **Compresses** | Conversation history (input) | LLM responses (output) | Shell command output (input) |
| **Technique** | Dedup + condense + LLM summarize | Prompt engineering ("be terse") | Per-command Rust filters |
| **Reduction** | 67-97% input tokens | ~65% output tokens | 60-90% tool output |
| **Target** | Any LLM app or tool (proxy mode) | AI coding agents | AI coding agents |
| **Integration** | One env var or one line of code | Config file install | Shell hook |

**These tools are complementary.** Use all three together for maximum savings:
- **RTK** compresses tool results as they enter the conversation
- **Caveman** makes the LLM respond more concisely
- **tokensqueeze** compresses the accumulated history before each API call

## Measuring Your Savings

tokensqueeze reports savings on every call:

```python
result = squeeze(messages, use_llm=True)
print(result)                 # CompressionResult(5,724 → 507 tokens, saved 91.1%)
print(result.strategy_stats)  # per-strategy breakdown
```

For tracking overall token costs across your workflow, [ccusage](https://github.com/ccusage/ccusage) is a useful companion — it reads usage logs from coding agents and generates cost reports:

```bash
npx ccusage@latest
ccusage claude daily  # compare before and after adding tokensqueeze
```

## License

MIT
