Metadata-Version: 2.4
Name: contextcram
Version: 0.1.0
Summary: Fit anything into an LLM context window — a tiny, zero-dependency, priority-aware token-budget packer.
Project-URL: Homepage, https://github.com/Waelr1985/contextcram
Project-URL: Repository, https://github.com/Waelr1985/contextcram
Project-URL: Issues, https://github.com/Waelr1985/contextcram/issues
Project-URL: Changelog, https://github.com/Waelr1985/contextcram/blob/main/CHANGELOG.md
Author-email: Waelr1985 <waelr1985@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: agents,anthropic,context-window,llm,openai,prompt,prompt-engineering,rag,token-budget,tokens
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.10
Provides-Extra: tiktoken
Requires-Dist: tiktoken>=0.5; extra == 'tiktoken'
Description-Content-Type: text/markdown

# contextcram

[![PyPI version](https://img.shields.io/pypi/v/contextcram.svg)](https://pypi.org/project/contextcram/)
[![Python versions](https://img.shields.io/pypi/pyversions/contextcram.svg)](https://pypi.org/project/contextcram/)
[![CI](https://github.com/Waelr1985/contextcram/actions/workflows/ci.yml/badge.svg)](https://github.com/Waelr1985/contextcram/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**Fit anything into an LLM context window.** A tiny, zero-dependency, priority-aware
token-budget packer for RAG pipelines and agents.

Every RAG or agent app has the same problem: you have too much stuff — a system
prompt, chat history, retrieved documents, tool output — and a fixed token
budget. `contextcram` packs it all in *by priority*, truncating, trimming, or
dropping the least important pieces so the important ones always make it.

```python
from contextcram import Packer

packer = Packer(budget=8000)  # token budget

packer.add(system_prompt, priority="required")                 # never dropped
packer.add(chat_history, priority="high", strategy="trim")     # drop oldest turns
packer.add(retrieved_docs, priority="medium", strategy="drop") # all-or-nothing
packer.add(tool_output, priority="low", strategy="truncate")   # cut to fit

result = packer.fit()
print(result.text)            # assembled, in-budget context
print(result.used_tokens)     # e.g. 7840
print(result.dropped_names)   # what didn't make the cut
```

## Why

- **Zero dependencies.** Pure Python. Works out of the box with a fast
  characters-per-token heuristic; plug in `tiktoken` or any tokenizer when you
  need exact counts.
- **Framework-agnostic.** Use it with LangChain, LlamaIndex, the raw provider
  SDKs, or nothing at all.
- **Priority-aware.** You decide what survives a tight budget, not a blind
  truncate at the end.
- **Observable.** Every result tells you what was kept, truncated, and dropped.

## Installation

```bash
pip install contextcram
# optional: exact token counts via tiktoken
pip install "contextcram[tiktoken]"
```

## Strategies

When an optional item doesn't fully fit, its `strategy` decides what happens:

| Strategy         | Behavior                                              |
| ---------------- | ----------------------------------------------------- |
| `drop`           | Include the item whole, or not at all                 |
| `truncate`       | Cut from the end, keeping the head (default)          |
| `truncate_head`  | Cut from the start, keeping the tail                  |
| `trim`           | For list content: drop oldest segments first          |

`required` items are always kept; if they alone exceed the budget, a
`BudgetExceeded` error is raised.

## Exact token counts

```python
from contextcram import Packer, tiktoken_tokenizer

packer = Packer(budget=8000, tokenizer=tiktoken_tokenizer("gpt-4o"))
```

Or wrap any tokenizer with `CallableTokenizer(lambda s: len(my_encode(s)))`.

## Priorities

Use the named levels `"required"`, `"high"`, `"medium"`, `"low"`, or pass any
integer (higher is kept first):

```python
packer.add(text, priority=42, strategy="truncate")
```

## Development

```bash
git clone https://github.com/Waelr1985/contextcram.git
cd contextcram
uv sync
uv run pytest
uv run ruff check .
uv run mypy
```

## License

MIT
