Metadata-Version: 2.4
Name: aether-context
Version: 0.3.0
Summary: Unlimited Context — virtual memory for an LLM's attention. Local-first, numpy-only core.
Author: Aether AI
Maintainer: Aether AI
License: Apache-2.0
Project-URL: Homepage, https://github.com/AetherAI3/Unlimited-Context-LLM
Project-URL: Repository, https://github.com/AetherAI3/Unlimited-Context-LLM.git
Project-URL: Documentation, https://github.com/AetherAI3/Unlimited-Context-LLM#readme
Project-URL: Issues, https://github.com/AetherAI3/Unlimited-Context-LLM/issues
Project-URL: Changelog, https://github.com/AetherAI3/Unlimited-Context-LLM/blob/main/CHANGELOG.md
Keywords: llm,context,retrieval,ollama,local,rag,agents
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: <3.15,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE.md
Requires-Dist: numpy<3.0,>=1.24
Provides-Extra: ollama
Provides-Extra: llamacpp
Requires-Dist: llama-cpp-python>=0.2; extra == "llamacpp"
Provides-Extra: hf
Requires-Dist: transformers>=4.40; extra == "hf"
Requires-Dist: torch>=2.2; extra == "hf"
Provides-Extra: fast
Requires-Dist: hnswlib>=0.8; extra == "fast"
Provides-Extra: all
Requires-Dist: aether-context[fast,hf,llamacpp]; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Requires-Dist: mypy>=1.10; extra == "dev"
Dynamic: license-file

<div align="center">

# ⚡ Unlimited Context LLM

**Give your AI superpowers with unlimited context for [Ollama](https://ollama.com)** — a **billion+ token memory** for any LLM. Local-first, on your own machine, free.

<img width="537" height="405" alt="Unlimited Context" src="https://github.com/user-attachments/assets/79758729-ead7-42ca-9784-831cae68ef06" />

[![License](https://img.shields.io/badge/License-Apache_2.0-06b6d4?style=flat-square)](LICENSE)
[![Python](https://img.shields.io/badge/Python-3.10%2B-14b8a6?style=flat-square&logo=python&logoColor=white)](https://www.python.org)
[![Built by Aether](https://img.shields.io/badge/Built_by-Aether-7c3aed?style=flat-square)](https://aethersystems.net)
[![Local-first](https://img.shields.io/badge/Local--first-100%25_offline-0ea5e9?style=flat-square)](#what-you-get)
[![PRs welcome](https://img.shields.io/badge/PRs-welcome-22c55e?style=flat-square)](CONTRIBUTING.md)
[![Stars](https://img.shields.io/github/stars/AetherAI3/Unlimited-Context-LLM?style=flat-square&logo=github&color=eab308)](https://github.com/AetherAI3/Unlimited-Context-LLM/stargazers)

**An open project from [Aether](https://aethersystems.net)** · Apache-2.0 · [Install](#quickstart)

<p align="center">
  <a href="#the-shape-of-it">The shape of it</a> ·
  <a href="#the-proof">Proof</a> ·
  <a href="SAFETY.md">Safety</a> ·
  <a href="USE_POLICY.md">Use policy</a> ·
  <a href="#the-problem">Problem</a> ·
  <a href="#how-it-works">How it works</a> ·
  <a href="#mpo-the-context-chain">MPO chain</a> ·
  <a href="#pick-your-memory-size">Memory size</a> ·
  <a href="#the-math-per-tier">The math</a> ·
  <a href="#ram-footprint">RAM</a> ·
  <a href="#common-commands">Commands</a> ·
  <a href="#the-aether-coding-terminal">Terminal</a> ·
  <a href="#quickstart">Install</a> ·
  <a href="#citation">Cite</a>
</p>

</div>

---

> **Your context window didn't get bigger. Its *reach* did.**
> Unlimited Context is virtual memory for an LLM. The model keeps its small window; the engine keeps a vast store on your disk and pulls the *right slice* back in while the model reasons. A small local model stays coherent across runs that would blow past any context window.

> ⚠️ **Giving a model durable memory is powerful — read the safety measures first.** Real
> concerns (runaway agents, grounding drift, an agent's own notes becoming its rules) and what
> we do about them: **[Aether AI — Ethical & Safety Measures](SAFETY.md)**. The project is open
> under Apache-2.0 and governed by the **[Acceptable Use Policy](USE_POLICY.md)** — these are not
> recommendations. Anyone using the project must comply with its terms, and by using the project you
> already agree to and are bound by them.

---

## The shape of it

Four steps, start to reach:

1. **Pick a model** — any local model (Ollama, llama.cpp, Hugging Face) or your own API-backed one.
2. **Start a session** — one object wraps the model and its memory.
3. **Allocate disk** — choose a pool size; that disk *is* the memory.
4. **Reach 1,000×+ further** — overflow is encoded to the pool and the exact slice is recovered the moment the model needs it.

That's it. A 5 GB pool gives a small model ~1.16B tokens of reach — about **9,000×** a 128K window — on your own machine, offline.

---

## The problem

Long agentic runs all die the same way. The model fills its window, starts **compressing** its own history, silently drops the one detail that mattered three steps ago — and drifts. You've seen it: the runaway PR, the agent that rewrites a function it already wrote, the build that falls apart at hour two. Bigger windows just delay it, and a crammed 1M-token window **rots in the middle** anyway.

The fix isn't a bigger window — it's to stop throwing the overflow away. Instead of summarizing what spills over, Unlimited Context **encodes** it to a local pool on your disk and **recovers** the right slice exactly when it's needed. Nothing load-bearing is silently lost.

<p align="center"><strong>Compress &amp; forget ✗ &nbsp;→&nbsp; Encode &amp; recover ✓</strong></p>

## How it works

It's **virtual memory, for attention.** Map it to an OS and it clicks:

| OS | Unlimited Context |
|---|---|
| RAM | the **resident window** the model sees now (small, fast) |
| Disk | the **context pool** — your encoded memory (~5 GB ≈ ~1B tokens) |
| Pager | the **slice loader** — prefetches the next slice from what the model is reasoning about *right now* |
| Page-replacement | the **retention policy** — useful slices *stay*, stale ones *fade*, anything relevant again comes back |

All of it runs while the model generates, so reaching the pool adds no wall-clock. → full explainer in [`docs/how-it-works.md`](docs/how-it-works.md).

## What you get

- 🧠 **Unbounded reach** — ~1B tokens of encoded context in ~5 GB on disk; the model reaches it in slices.
- 🧩 **MPO context chain** — recall pulls the whole connected thread, not isolated nearest-neighbors (5× connected-context recall in-bench).
- ⚡ **Zero added latency** — the pager runs *concurrently* with generation, hidden behind the model's own thinking.
- 🪟 **Curated beats crammed** — a small, relevant resident window outperforms a stuffed one (no lost-in-the-middle) — and costs less.
- 🔒 **Local-first** — your context never leaves your machine. Free storage, full privacy, works offline.
- 🤖 **Any model** — Llama, Qwen, Mistral, Phi — via Ollama, llama.cpp, or HF, or your own API-backed model.
- 📉 **Coherence you can *measure*** — ship the head-to-head: same model, engine on vs off, watch the drift rate fall off a cliff.

## Pick your memory size

First run drops you into a slider — pick how much your model gets to remember:

```text
$ aether-context init
──────────────────────────────────────────────────────────────────
  ⚡ choose your context pool          encoded reach · not a window
──────────────────────────────────────────────────────────────────
  ▸  5 GB   ████░░░░░░░░░░░░   ~1.16B tokens   a big project   (floor)
     10 GB  ████████░░░░░░░░   ~2.33B tokens   a large monorepo + docs
     15 GB  ████████████░░░░   ~3.49B tokens   multiple repos / long runs
     20 GB  ████████████████   ~4.65B tokens   massive corpus / power user
──────────────────────────────────────────────────────────────────
  reach ≈ pool_GB × 233M tokens     custom: --pool 12  (any size ≥ 5 GB)
  ↑/↓ slide      ↵ confirm

  pool [5]: 10
  ✓ 10 GB  →  your model can now reach ~2.33 billion tokens
```

## What that buys you in coding time

The real win isn't the token count — it's that the wall disappears. A typical ~128K context window fills after well under an hour of active agent work, then starts compacting and forgetting. A 5 GB pool is **~9,000× bigger**.

Rough ballpark — assuming a busy coding agent encodes ~300K–1M keep-worthy tokens/hour (chatty swarms burn more, careful single agents less):

| Pool | Reach | Active autonomous coding before it even fills\* |
|:----:|:-----:|:------------------------------------------------|
| **5 GB** | ~1.16B | **~1,200–3,900 hrs** — weeks of nonstop building |
| 10 GB | ~2.33B | ~2,300–7,800 hrs |
| 15 GB | ~3.49B | ~3,500–11,600 hrs |
| 20 GB | ~4.65B | ~4,700–15,500 hrs |

For color: 5 GB of reach ≈ ~100M lines of code, or a shelf of ~8,000 books — you won't fill it in one sitting.

<sub>\* Rough order of magnitude. Because the retention policy fades stale slices, the pool never hard-stops anyway — it just keeps what's relevant. Run a build as long as you want; it won't lose the plot. The per-session RAM math is in [RAM footprint](#ram-footprint) below.</sub>

## Running many sessions

Running more than one agent? How the pool is shared is the single biggest RAM lever:

- **`--pool-mode shared`** — one pool, one index, all sessions reach the same memory. The index is paid **once**; each extra session adds only ~30 MB, so RAM barely moves as you add sessions. Best for related work (same project) or max concurrency on a small machine. Trade-off: sessions can see each other's context (no isolation).
- **`--pool-mode separate`** *(default)* — each session gets its own pool + index, fully **isolated and private**. Clean, but you pay one index **per session**, so RAM scales with `N × pool`. Best for unrelated tasks or when isolation matters.

**How many actually fit:**

| Pool | 8 GB · shared | 8 GB · separate | 16 GB · shared | 16 GB · separate |
|------|---------------|-----------------|----------------|------------------|
| 5 GB  | dozens¹ | **~13** | dozens¹ | **~33** |
| 10 GB | dozens¹ | **~7**  | dozens¹ | **~18** |
| 15 GB | dozens¹ | **~4**  | dozens¹ | **~12** |
| 20 GB | dozens¹ | **~3**  | dozens¹ | **~9**  |

<sub>Reserves: ~2.5 GB held back on an 8 GB machine, ~6 GB on 16 GB — the rest stays for your OS and editor. ¹ With a shared pool, RAM stops being the limit (50–70+ sessions fit); you're bounded by CPU and good sense, not memory.</sub>

## The math, per tier

Derived, not vibes:

| Pool | Slices | Encoded reach | Slider |
|------|--------|---------------|--------|
| **5 GB** *(floor)* | 2.27M | **~1.16B tokens** | `████░░░░░░░░░░░░` |
| 10 GB | 4.55M | **~2.33B tokens** | `████████░░░░░░░░` |
| 15 GB | 6.82M | **~3.49B tokens** | `████████████░░░░` |
| 20 GB | 9.09M | **~4.65B tokens** | `████████████████` |

How those numbers come out: ~2.2 KB per slice (a 256-dim vector + compressed text + metadata) ÷ 512 tokens per slice → **~455K slices/GB → ~233M tokens of reach per GB**. So `reach ≈ pool_GB × 233M`. 5 GB is the floor; bump anytime with `aether-context --pool 20`.

> **Honest:** that's encoded **reach**, retrieved in slices — not a bigger attention window, and it rides on retrieval hit rate. A bigger pool buys more reachable codebase/corpus *per session* — not more concurrent sessions (those are RAM-bound, ~30 on 8 GB either way).

---

## The proof

Not a synthetic micro-benchmark — a **real, paid, end-to-end run.** A reasoning model
(`deepseek-v4-pro`, via OpenRouter) driven through a **40-turn agent session that overflows its
window** (2,000-token window, 60 real `microsoft/vscode` issues), measured **engine off vs on** —
one live run, **$0.19**, 2026-06-14.

- **The model stops forgetting.** Recall of early facts after they fall out of the window:
  **0.15 → 1.00.** The baseline drifts and forgets; the engine holds every early fact — zero drift.
- **Failure turns into success on the real work.** Tasks completed correctly: **3 / 20 → 20 / 20.**
  The job is only done right *with* the engine.
- **Cheaper, not just better.** **−24%** total cost, **−54%** in the back half — the engine sends a
  compact recalled slice instead of dragging the whole transcript into every call.

<p align="center">
  <img alt="Cumulative cost and recall coherence vs turn — engine off vs on" width="780"
       src="docs/benchmarks/artifacts/2026-06-14-deepseek-v4-pro/api_eval_plot.png">
</p>

| Metric | Off (baseline) | On (engine) | Change |
|---|:---:|:---:|:---:|
| **Recall coherence** (early facts still correct) | 0.15 | **1.00** | **6.7×** |
| **Work outcome** (tasks done right) | 3 / 20 | **20 / 20** | **3 → 20** |
| **Cost — full session** | $0.0711 | **$0.0542** | **−24%** |
| **Cost — back half (recall phase)** | $0.00117/turn | **$0.00053/turn** | **−54%** |

**Committed data:** [full write-up](docs/benchmarks/2026-06-14-deepseek-v4-pro-session-eval.md) ·
[raw artifacts](docs/benchmarks/artifacts/2026-06-14-deepseek-v4-pro/) (`api_eval_results.json`,
`api_eval_series.csv`, `api_eval_plot.png`, `RESULTS.md`) · reproduce with
`python -m bench.api_eval --model deepseek/deepseek-v4-pro --repo microsoft/vscode --arms off,on,on_chain --plot`

<sub>**Scope, honestly:** this measures the **engine** (retrieve-on-overflow memory), not the MPO
chain — on this single-fact recall task the chain **ties** plain recall (both 1.00); its multi-slice
edge is **synthetic-only so far** (`bench/chain_recall.py`: connected-context recall 0.15 → 0.78),
with the live `thread` run **pending**, not yet claimed. The 2,000-token window is deliberately tiny
to force overflow, so a realistic window shows a smaller (still real) gain. N = 20 recall turns,
single run.</sub>

---

## RAM footprint

The engine stays light: **vectors live on disk (mmap'd)** — only the small HNSW index graph and a hot working set are ever resident. So RAM is a predictable formula, not a mystery:

```
RAM  ≈  ~180 MB   base (engine + shared static encoder)
      +  ~29 MB   per GB of pool   (resident index)
      +  ~30 MB   per active session
```
## MPO: the context chain

Plain semantic search returns isolated nearest-neighbors — the single closest slices, ripped out of the thread they belonged to. Recall a fact and you often miss the three slices around it that made it make sense.

The **MPO context chain** fixes that. It links the session's slices into one connected structure, so when cosine pulls an entry slice, the chain **pulls in the slices most coupled to it** — widening the working set with the *connected thread*, not stray hits. Cosine is still the retrieval mechanism; the chain **assists** it.

The chain is **Aether-tuned, deterministic, and fully local** — no training, no network. It's purely **additive**: it only ever *adds* connected context, never blocks or replaces a hit, and on any hiccup it falls back cleanly to plain cosine.

In a planted-thread benchmark, this lifts connected-context recall from **0.15 (cosine alone) to 0.78** — over 5× more of the right thread in the window. On by default:

```python
Session(model="ollama/qwen2.5", pool_gb=10)                 # chain on by default
Session(model="ollama/qwen2.5", pool_gb=10, mpo_chain=False) # plain cosine
```
```bash
aether-context run "..." --no-mpo-chain                      # disable for one run
```

**Resident index cost by pool size:**

| Pool | Index RAM (resident) |
|------|----------------------|
| 5 GB | ~146 MB |
| 10 GB | ~291 MB |
| 15 GB | ~436 MB |
| 20 GB | ~582 MB |

(The encoder is always shared — stateless, ~31 MB, loaded once. Only the pool/index differs.)

> **TL;DR.** **Shared pool → RAM is not your limit** — spin up as many sessions as your CPU allows. **Separate pools → one index each**, so plan on ~3 (20 GB) to ~13 (5 GB) sessions on 8 GB, roughly double at 16 GB. A bigger pool always buys **reach**, never more sessions. Need more headroom? Shrink the pool. (`--index tiered` is reserved for a future paged-graph index and currently runs the flat index — it does not yet reduce resident RAM.)

## Common commands

<div align="center">
  <img width="880" alt="Coding time per pool size" src="https://github.com/user-attachments/assets/af626850-96b1-43a2-91fd-b5162bc21e5a" />
</div>

| Command | What it's for |
|---|---|
| `aether-context setup` | **Start here.** Guided first run: size the pool, check your model, verify the engine. |
| `aether-context init` | Pick your pool size — the on-disk storage slider — on first run. |
| `aether-context run "<task>" --no-mpo-chain` | Run with the MPO context chain disabled (plain cosine). |
| `aether-context run "<task>"` | One-shot a task with full reach, then print the result. |
| `aether-context chat` | Open an interactive session; type `/status` anytime, `/clear` to reset. |
| `aether-context status` | See pool size, slices used, reach, and hit rate at a glance. |
| `aether-context doctor` | Check Ollama, your model, disk, and RAM before a long run. |
| `aether-context --pool 20` | Resize the pool anytime (non-destructive re-index). |

> **Tip:** run `aether-context doctor` first — it catches the three things that ever go wrong (Ollama down, model not pulled, not enough disk) and prints the exact fix.

## The `aether` coding terminal

**`aether`** is an open-source agentic **coding terminal** that runs on the Unlimited Context
engine. It's **local-first**: turns run on your local **[Ollama](https://ollama.com)** by default
(no account, no network); sign in and they switch to the **Aether cloud API**.

It ships as its own package, not as part of `aether-context`:

```bash
pip install aether-agent      # or: npm install -g aether-agents
```

Source lives at [AetherAI3/aether-agent](https://github.com/AetherAI3/aether-agent). The
`aether_agent/` directory in *this* repo is the Python-native twin — same commands, same backend,
same tools — kept here for development, and deliberately **not** published from this package:
PyPI's `aether-agent` already owns that import path and the `aether` command, so shipping a
second copy would silently overwrite it wherever both are installed.

| Command | What it does |
|---|---|
| `aether` | Open the interactive REPL (local Ollama by default). |
| `aether "<prompt>"` | One-shot turn, streamed. |
| `aether code "<task>"` | Autonomous coding run on the Unlimited Context brain (test-gated, git-checkpointed). |
| `aether auth login` | Sign in (`--token <t>` or `--username/--password`) for the hosted API (then set `backend auto\|cloud`). |
| `aether auth status \| logout \| token` | Show / clear / print the stored credential. |
| `aether models` | List models available to your tier. |
| `aether config [show\|get <k>\|set <k> <v>]` | Local settings, incl. `backend = local (default)\|auto\|cloud`. |

**Slash commands** (inside the REPL): `/help` · `/models` · `/model <tag>` · `/agents` · `/agent <id>` · `/tier` · `/audit [n]` · `/web <query>` · `/clear` · `/exit`.

**Web tools** — the agent can reach the web on any backend: `web_search` (DuckDuckGo, no key) and `web_fetch` (URL → readable text, SSRF-guarded).

**Backend** — `local` (the default) runs every turn on your own Ollama; nothing leaves your machine and no account is needed. Opt into the hosted Aether API with `aether config set backend auto|cloud` (or `AETHER_BACKEND=auto`) plus `aether auth login` — `auto` uses the cloud only when you're signed in and falls back to local otherwise. The hosted API is the maintainer's own instance, so a fresh clone never calls it.

**Smoke test** — from a clone of this repo, with Ollama up, `python -m aether_agent.smoke` runs the SSRF guard, a real local turn, a web search + fetch, and the cloud path (when signed in), printing `PASS`/`SKIP`/`FAIL` (exit non-zero only on a real failure — missing Ollama/network/sign-in are skips).

## Quickstart

```bash
pip install aether-context
aether-context setup
```

Prefer npm? Same software, same release — the npm package is a launcher that installs the
Python engine into a private virtualenv for you (it needs Python 3.10+ on your PATH):

```bash
npx aether-context setup
```

`setup` sizes the pool, checks for a local model, and verifies the engine end to end. It works
with no daemon, no network and no model pulled — the check runs against the built-in mock model.

<details>
<summary>Other install routes</summary>

```bash
# Straight from source, always the latest main:
pip install git+https://github.com/AetherAI3/Unlimited-Context-LLM.git

# Isolated, if you only want the CLI:
pipx install aether-context
```

The distribution name is **`aether-context`** — `pip install unlimited-context` is not this
package.
</details>

```python
from aether_context import Session

s = Session(model="ollama/qwen2.5", pool_gb=5)
s.run("Build me a full-stack weightlifting tracker app.")
# runs long. stays coherent. walk away.
```

That's the whole thing. One small model, one command, a billion tokens of reach behind it.

## Honest about the word "unlimited"

"Unlimited" means **reach, not attention.** Your model keeps its native window — we make it *reach* a billion-token pool in slices, via fast retrieval. The whole thing rides on retrieval **hit rate**; when it's high (and the loader is built to keep it high), the pool feels like one seamless context. The measured proof of all this is up top — see [The proof](#the-proof).

## Citation

If Unlimited Context helps your work, please cite it. Built and maintained by **Aether AI**.

```bibtex
@software{unlimited_context_2026,
  title        = {Unlimited Context (aether-context): virtual memory for LLM attention},
  author       = {Barrante, Brandon},
  organization = {Aether AI},
  year         = {2026},
  url          = {https://github.com/AetherAI3/Unlimited-Context-LLM},
  license      = {Apache-2.0}
}
```

GitHub's "Cite this repository" button reads [`CITATION.cff`](CITATION.cff) directly.

## ⭐ Star, share, contribute

If this gave your local model superpowers, **drop a star** — it's how other people find it. **PRs and issues are welcome** — see [CONTRIBUTING.md](CONTRIBUTING.md). Runnable examples live in [`examples/`](examples/) — start with [`quickstart.py`](examples/quickstart.py), then [`coding_agent.py`](examples/coding_agent.py).

## License

**Apache-2.0.** Use it, fork it, ship it in your product.

---

<div align="center">

Built by **Aether AI** · [Aether](https://aethersystems.net)

<img width="880" alt="Unlimited Context" src="https://github.com/user-attachments/assets/4b7eef9a-8b1c-4dc7-b926-771ce53ed04d" />

*Unbounded reach for the model you already run.*

</div>
