Metadata-Version: 2.4
Name: sillage
Version: 1.0.1
Summary: Gradient-free test-time memory and learning for frozen language models
Author-email: Abderrahmane Sghairi <sghairipro63@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/riscoss63/sillage
Project-URL: Papers, https://doi.org/10.5281/zenodo.22079016
Keywords: language-models,associative-memory,hebbian,test-time-learning,fast-weights,gradient-free,continual-learning,vsa,hyperdimensional
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: torch>=2.0
Requires-Dist: transformers>=4.51
Provides-Extra: research
Requires-Dist: scipy>=1.10; extra == "research"
Requires-Dist: matplotlib>=3.7; extra == "research"
Dynamic: license-file

# Sillage

**Your language model forgets everything. Sillage gives it a memory — and a
way to keep learning — in a fixed handful of megabytes, with no gradients and
no index that grows.**

> *sillage* (n., French) — the trace left behind by something that has passed:
> a ship's wake, a scent in a room. What a model keeps of what it read.

A frozen language model reads your documents, remembers them, and predicts
better next time. No gradients, no fine-tuning, no vector database. One
Hebbian matrix written as the model reads, a semantic tier routed by
confidence, a cold store that consolidates by surprise, and a rank-16 adapter
on the readout — four mechanisms, four preprints, one command-line tool.
Everything runs on a laptop CPU, and on a GPU when there is one.

![demo](https://raw.githubusercontent.com/riscoss63/sillage/main/figs/demo.gif)

## Install

```bash
pip install sillage
```

## Use

```bash
sillage index notes.md                # instant: no model, already queryable
sillage ask "what did the report say?"

sillage read notes.md                 # memorize it (CPU: ~8 min per 10k tokens)
sillage complete "The report said"    # generate WITH the memory
sillage status                        # what it knows, tier by tier
sillage chat                          # ask and generate in one session
sillage forget --all
```

```console
$ sillage read preprint_v1.txt                    # Monday, memory empty
read preprint_v1.txt: 5859 tokens in 4.9 min | PPL 12.14 -> 11.83 (adapter) -> 11.71 (+memory)

$ sillage read preprint_v2.md                     # Tuesday, a new process
read preprint_v2.md: 11041 tokens in 9.5 min | PPL 10.68 -> 9.82 (adapter) -> 5.39 (+memory)
```

Three numbers per file: what the frozen model alone predicts, what the
rank-16 adapter adds, and what the memory of everything read so far adds on
top. The state lives in `./.sillage`, survives restarts, and never grows:
7.4 MB with GPT-2, 25 MB with Qwen3, the same after one document or ten
thousand. It never learns from its own generations — only from what you give
it to read.

From Python:

```python
from sillage import Sillage

s = Sillage(model="gpt2")          # any causal LM; omit it and the state
                                   # says which model it belongs to
s.read("notes.md")                 # read, memorize, index -- then save
s.ask("what did the report say?")  # exact passages, nothing generated
print(s.complete("The report said"))
```

## How much better, exactly?

On 36k tokens of technical text the model had never seen (frozen GPT-2 124M,
every system tuned identically on a held-out prefix, 95 % bootstrap CIs):

| system | perplexity | change | memory used |
|---|---|---|---|
| frozen GPT-2 | 31.2 | — | 0 |
| \+ RAG-style retrieve & rescore | 29.9 | −4 % | corpus + index |
| \+ kNN-LM, **unbounded** store | 23.6 | −24 % | 55 MB, grows forever |
| \+ **this memory** (fixed) | **19.2** | **−38 %** | **4.2 MB, constant** |
| \+ memory **and** fast weights | **16.6** | **−47 %** | 7.4 MB, constant |

The fixed 4.2 MB memory beats the unbounded datastore it was designed to
approximate — paired bootstrap **P = 1.000**, replicated over 5 random seeds
and on a second model (Qwen3-0.6B).

**Where it does not win:** on long, low-repetition narrative text, kNN-LM
still beats it. This memory captures verbatim recurrence; that is its regime,
and it is measured and published rather than hidden.

## Any causal language model, on CPU or GPU

```bash
sillage read notes.md --model qwen                          # shortcut
sillage read notes.md --model HuggingFaceTB/SmolLM2-135M    # any hub id
sillage read notes.md --model ./my-finetuned-llama          # any local folder
sillage read notes.md --device cuda                         # if you have one
```

Verified across three architectures (GPT-2, Qwen3, GPT-NeoX). Three things
worth knowing before pointing it at a new model:

- **The readout tunes itself.** For a model nobody has tuned, it calibrates on
  a rolling window of what you read, following the papers' protocol: the
  winner governs the *next* read, never the one it was fitted on. For the two
  models the papers did tune, their published settings are kept — refitting
  those on a cold memory measurably loses (+0.109 against +0.120 nats).
- **A memory lives in one model's token space.** Give each model its own
  `--state` directory; the state remembers which model it belongs to and
  refuses to be opened by another.
- **The GPU only does the frozen forward passes.** `--device cuda` moves the
  model; the mechanisms stay in numpy on the CPU, where they belong — they are
  rank-1 updates, not matrix multiplications. Defaults to the GPU when there
  is one.

Requires Python 3.10+, `numpy`, `torch` and `transformers`. Nothing else, and
no network at all once the frozen model is cached.

## The four preprints

| # | title | DOI |
|---|---|---|
| 1 | Sillage: Surprise-Gated Amplitude Memory for Frozen Language Models | [10.5281/zenodo.22079016](https://doi.org/10.5281/zenodo.22079016) |
| 2 | Route the Scores, Not the Keys | [10.5281/zenodo.22079444](https://doi.org/10.5281/zenodo.22079444) |
| 3 | One Signal, Three Tiers | [10.5281/zenodo.22079471](https://doi.org/10.5281/zenodo.22079471) |
| 4 | Memory Remembers, Fast Weights Adapt | [10.5281/zenodo.22079481](https://doi.org/10.5281/zenodo.22079481) |

Sources, figures, every number as committed JSON, the reproduction pipeline
and the three negative results are on GitHub:
**<https://github.com/riscoss63/sillage>**

## Citation

```bibtex
@article{sghairi2026sillage,
  title  = {Sillage: Surprise-Gated Amplitude Memory
            for Frozen Language Models},
  author = {Sghairi, Abderrahmane},
  year   = {2026},
  doi    = {10.5281/zenodo.22079016},
  url    = {https://doi.org/10.5281/zenodo.22079016}
}
```

MIT licensed.
