Metadata-Version: 2.4
Name: streamft
Version: 0.8.2
Summary: Fine-tune models that do not fit: stream NF4 weights from disk, one transformer block at a time.
Author-email: Gokhan Cava <gokhancava@gmail.com>
License: Apache-2.0
Project-URL: Homepage, https://github.com/byjameson/stream-finetune
Project-URL: Issues, https://github.com/byjameson/stream-finetune/issues
Project-URL: LinkedIn, https://www.linkedin.com/in/g%C3%B6khan-cava-874a32157/
Keywords: fine-tuning,qlora,nf4,low-vram,lora,llm
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: torch
Requires-Dist: transformers>=4.40
Requires-Dist: safetensors
Requires-Dist: tokenizers
Requires-Dist: accelerate
Requires-Dist: datasets
Requires-Dist: huggingface-hub
Requires-Dist: numpy
Requires-Dist: psutil
Requires-Dist: bitsandbytes
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: peft>=0.21; extra == "dev"
Requires-Dist: scikit-learn; extra == "dev"
Requires-Dist: scipy; extra == "dev"
Requires-Dist: joblib; extra == "dev"
Dynamic: license-file

# Atom Zero — a 49.2M local assistant that cites its sources

[![release](https://img.shields.io/badge/release-0.8.2-2b6f6a)](#) [![params](https://img.shields.io/badge/core-49.2M-4b6f9e)](#) [![tests](https://img.shields.io/badge/tests-392-6b4a8a)](#) [![license](https://img.shields.io/badge/license-Apache--2.0-555)](#)

> A 49 million parameter model that says “I don’t know”, retrieves **your** documents with citations, keeps personal notes and tasks, and asks before it researches. No cloud, no telemetry, no giant model.

![Atom Zero architecture](https://raw.githubusercontent.com/byjameson/stream-finetune/main/docs/figures/atomzero_architecture.png)

## 60-second demo

```powershell
git clone https://github.com/byjameson/stream-finetune.git
cd stream-finetune
pip install streamft==0.8.2
powershell -ExecutionPolicy Bypass -File scripts/quickstart_atomzero.ps1
```

The package is on PyPI: <https://pypi.org/project/streamft/0.8.2/>. The demo
checkpoint still comes from the repository/tag.

Full guide: [`docs/QUICKSTART_ATOMZERO.md`](docs/QUICKSTART_ATOMZERO.md). The script seeds a small demo corpus (launch code, warranty, travel, visitor badge), starts the local server, and opens the browser. Try:

```text
What is the launch code?                         -> cited local answer
What is the population of Paris?                 -> "I don't have a source"; gap recorded
Propose curiosity topics                         -> pending research, approval required
Add task buy milk / My name is Alex              -> local tools + memory
```

Optional stronger local chat (RAG/tools stay local):

```powershell
powershell -ExecutionPolicy Bypass -File scripts/quickstart_atomzero.ps1 -WithProvider
```

## What it is

- **Source-cited RAG** over your own text files, with a hybrid FTS + MiniLM retriever.
- **Deterministic tools**: arithmetic, time, tasks, name/likes memory, document search.
- **Honest missing-source path**: no note means “I don’t have a source”, not an invented answer.
- **Curiosity queue**: unanswered questions become pending research; you approve before any fetch or teacher call.
- **Reviewed self-improvement**: feedback → human gate → LoRA adapter → rollback. RAG notes do not change weights; the model does not autonomously self-modify.
- **Optional provider**: plug in a local Ollama/OpenAI-compatible model for general chat.
- **No telemetry, no hosted service**; all state is local SQLite/JSON.

## Honest benchmark snapshot

| check | result |
|---|---:|
| raw 49.2M, no RAG/tools | **2/10** |
| product with RAG/tools | **21/21** |
| behavior checks | **16/16** |
| source-cited retrieval top-1 | **1.000** (FTS 0.850) |
| test suite | **392 passed** |

The 21/21 is the **product**, not the bare model. See the [model card](docs/MODEL_CARD.md), [benchmarks](docs/BENCHMARKS.md), [curiosity flow](docs/CURIOSITY.md), [capability ladder](docs/POTENTIAL.md), and [launch notes](docs/LAUNCH.md). The self-improvement boundary is: RAG notes (no weights) → reviewed LoRA (weights) → manual full training.

## Published

- GitHub release: <https://github.com/byjameson/stream-finetune/releases/tag/v0.8.2>
- PyPI: <https://pypi.org/project/streamft/0.8.2/>
- Hugging Face: <https://huggingface.co/byjameson/atom-zero-49m>
- ModelScope: <https://modelscope.cn/models/byjameson/atom-zero-49m>
- Launch article: <https://dev.to/mycute_97b39c4e34027570f9/atom-zero-a-49m-local-assistant-that-says-i-dont-know-1p7m>

## Contact

Questions, bug reports and collaboration:

- **Email:** gokhancava@gmail.com
- **LinkedIn:** <https://www.linkedin.com/in/g%C3%B6khan-cava-874a32157/>
- **GitHub issues:** <https://github.com/byjameson/stream-finetune/issues>

## streamft: fine-tune a 32B model on a 6 GB laptop GPU

[![PyPI](https://img.shields.io/pypi/v/streamft)](https://pypi.org/project/streamft/) [![try it](https://img.shields.io/badge/try-will%20it%20fit%3F-6ee7b7)](https://huggingface.co/spaces/byjameson/will-it-fit)

A small training engine that fine-tunes models that **do not fit in VRAM**. The NF4
weights are mmap'd from disk, one transformer block is materialised on the GPU at a
time, and the backward recomputes through it. The base stays frozen; a LoRA adapter is
trained.

**Try it without installing anything:** [Will it fit?](https://huggingface.co/spaces/byjameson/will-it-fit)
— type a model id and a VRAM size; it says whether the whole model fits and shows the
measured speed where a measurement exists (and says so where it doesn't).

**7B end-to-end on 6 GB:** [train an adapter, serve it on the 4-bit base, call it through
`/v1/chat/completions`](docs/DEMO_7B.md) — 20 steps in 88.6 s, 5.65 GB peak, 32 tokens in
3.14 s. This is the concrete “it works” scenario, not a benchmark.

## Atom Zero runtime details

**Atom Zero — local daily assistant (from scratch, 50M):** [tasks, personal
notes and source-cited document lookup](docs/TINY_TRANSFORMER_CHAT.md) — start
with `powershell -ExecutionPolicy Bypass -File scripts/demo_all.ps1`. The
Python package and CLI remain `streamft`; Atom Zero is the product name. Paste or
upload your own UTF-8 documents over HTTP; the response keeps source, title and
sha256. A reviewed research queue (browser + API/CLI) fetches only
user-approved Wikipedia topics on a schedule. Unanswered questions become
knowledge gaps; **Propose curiosity topics** turns them into pending research
items, and nothing runs until you approve it. See
[`docs/CURIOSITY.md`](docs/CURIOSITY.md). ChatGPT/other answers can be
pasted manually as unverified teacher notes with a source URL, or an explicit
local [Ollama](docs/TEACHER_OLLAMA.md)/OpenAI-compatible provider can answer on request or synthesize
approved research topics. Teacher notes start unverified and can be marked
verified by a named reviewer with a timestamp. Unknown questions become
**knowledge gaps**; approving a proposed topic lets the scheduled worker fetch
a source into RAG. Chat feedback can be reviewed, exported and sent through a
gated LoRA update; accepted `vNNNN` versions are recorded with metrics, can be
compared with `research/compare_adapters.py`, evaluated on a non-training
holdout, and rolled back with `learning_loop.py activate`. The active adapter
loads with `--adapter auto`. An opt-in `--safety-filter` runs exact rules first
and then a small local MiniLM linear ML head; it refuses explicit harmful
requests and filters output (multilingual red-team 72/72, held-out semantic
paraphrases 10/10, 0/26 benign false blocks). Missing model/head falls back to
exact rules; `--no-safety-ml` forces the rule baseline. It is **off by
default**: this is a local tool, not a hosted content policy. The optional
safety layer is an operator tool, not a moral gate.

An optional **local provider backend** can answer free-form chat with a
user-supplied Ollama/OpenAI-compatible model while RAG, tools and memory stay
local. The default remains the 50M model. See
[`docs/POTENTIAL.md`](docs/POTENTIAL.md) for the capability ladder and
[`docs/MODEL_CARD.md`](docs/MODEL_CARD.md) for the shareable benchmark summary.

Indexed sources and session memory have tested delete paths
(`docs/PRIVACY.md`). The current product checkpoint is the 49.2M scaled run
`checkpoints/transformer_behavior_50m_scaled` (40k steps); the original
`transformer_behavior_50m_long` is retained for comparison. The optional 368k
LoRA adapter and SQLite FTS5 memory run locally. RAG uses a hybrid candidate
stage: FTS plus cached chunk embeddings, with hashed character n-grams as the
dependency-free fallback. When `models/all-MiniLM-L6-v2` is present,
`--reranker auto` uses a local MiniLM neural pool; on the held-out authored
retrieval benchmark this reaches **1.000** top-1/citation vs **0.850** for FTS,
at about 8.8 ms per query on CPU. A prewarmed in-memory matrix makes the vector
stage **166x** faster than the linear scan on 3000 chunks with identical top-5;
above 256 chunks an IVF ANN index is used, with an exact cosine fallback and
recall@10 **1.000** on the 20k-vector benchmark. Facts stay in RAG; style/skill
updates go through the gate. It is not a general assistant. The project is open source; no hosted
service is operated. See the [decision pack](docs/RELEASE_DECISIONS.md),
[product audit](docs/PRODUCT_AUDIT.md),
[safety filter](docs/SAFETY.md),
[scaling plan](docs/SCALING_PLAN.md),
[data-scaling run](docs/DATA_SCALING.md) and
[research worker](docs/RESEARCH_WORKER.md) for what is still required.

**The smallest demo:** [130 KB AI](https://huggingface.co/spaces/byjameson/mini-ai) — a
news classifier that trains in your browser from 500 labelled examples and scores 78.3% on
the embedded 400 AG News rows (74.3% on a fresh 400; 82.8% at 2,000 examples on both). No
server, no GPU, no stored weights. Source: [`demo/mini-ai.html`](demo/mini-ai.html).

**Need accuracy instead of minimum bits?** [The 1.7 MB accuracy mode](https://byjameson-mini-ai.static.hf.space/max-ai.html)
uses 75,000 human-labelled rows and reaches **88.5%** on the same test set. Labels matter:
on the same 5,000 rows, 3B-Instruct teacher labels gave a **71.3%** student; human labels
gave **84.3%**.

**Make your own:** [Tiny AI Maker](https://byjameson-mini-ai.static.hf.space/maker.html)
takes a CSV of `text,label`, trains a classifier in the browser, and exports a single HTML
file, a **QR URL**, a favicon, a WAV or a **PDF model card** with the weights in `/Keywords`.
It also runs an optional **5-fold CV** against the 80/20 holdout, with out-of-fold
**Brier/ECE** calibration and a nested **temperature-scaling** check. No data leaves the
page.

**AI in a QR code:** [a 2.2 KB URL](https://byjameson-mini-ai.static.hf.space/qr.html)
carries an SMS spam classifier — 160 words, int8 weights, 93.5% on 1,115 held-out messages.
Scan the QR or open the link; the model is in the URL.

**The measured ladder:** [The Minimum-Bit Table](https://byjameson-mini-ai.static.hf.space/leaderboard.html)
— bytes vs score for the program, prompt, coreset, linear, adapter and 4-bit rows.

**Where 4-bit hurts:** [the adapter autopsy](https://byjameson-mini-ai.static.hf.space/diff.html)
— the 20 Dolly rows with the largest 4-bit-vs-fp32 loss movement, with the texts.

**A 5 MB fingerprint:** [the adapter watermark](https://byjameson-mini-ai.static.hf.space/watermark.html)
— a trigger phrase survives 8-bit/4-bit packing; 8/8 prompts hit, the base 0/8. Merging the
adapter into a task adapter is the hard part: the mark is 0/8 at a 0.50 merge and only
reliable at 0.90-1.00, and the exact threshold is seed- and token-dependent. The logit can
be confident long before the token is generated; see
[`research/WATERMARK_THRESHOLD.md`](research/WATERMARK_THRESHOLD.md). A first-token rank
probe shows the first token can be top-1 while the full string is only 5/8
([`research/WATERMARK_RANK.md`](research/WATERMARK_RANK.md)). Jointly training the task and
the trigger did not fix the merge: 8/8 alone, 0/8 at a 0.75 merge
([`research/WATERMARK_JOINT.md`](research/WATERMARK_JOINT.md)). Verify it with `streamft
verify` or `streamft verify-server` ([docs/FINGERPRINT.md](docs/FINGERPRINT.md)).

**AI in a spreadsheet:** [a 10.5 KB .xlsx](https://byjameson-mini-ai.static.hf.space/spreadsheet.html)
with the same 160 SMS words in cells; 90.6% formula accuracy, no macros.

**AI in a favicon:** [a 1.5 KB PNG](https://byjameson-mini-ai.static.hf.space/favicon.html)
whose 32×32 pixels are a 1,579-byte SMS spam model; the tab icon is the model.

**AI in a WAV:** [a 1.6 KB, 0.198-second sound](https://byjameson-mini-ai.static.hf.space/audio.html)
whose 8-bit PCM samples are the same model.

**AI in a PDF:** [a 3.7 KB model card](https://byjameson-mini-ai.static.hf.space/pdf.html)
whose visible page describes the model and whose `/Keywords` metadata carries the 1,579-byte
weights — one document is both the card and the classifier. The full SHA-256 is in
`/TinyAISHA256`, and the viewer checks it.

**AI in an SVG:** [a 4.9 KB vector card](https://byjameson-mini-ai.static.hf.space/svg.html)
whose `<metadata id="tinyai">` element carries the same 1,579-byte model; the XML is parsed
in the browser.

**AI in a MIDI:** [a 4.4 KB MIDI file](https://byjameson-mini-ai.static.hf.space/midi.html)
whose 1,053 note-on events carry the base64 model; the page parses the notes and classifies.

**The 130 KB AI in a PDF:** [a 175 KB PDF](https://byjameson-mini-ai.static.hf.space/pdf-130k.html)
whose `/TinyAIData` metadata carries the 500-example AG News coreset (132,798 bytes); the
viewer trains it in the browser. 78.3% embedded / 74.3% fresh on the 400-row test.

**Container survival:** [a measured matrix](https://byjameson-mini-ai.static.hf.space/survival.html)
of what happens after a re-save, re-encode, transcode or metadata strip. XLSX and single
HTML are the best archival carriers; PDF is not editable; PNG/WAV survive only lossless
pipelines; QR survives image abuse but not a dropped URL fragment.

Measured on one machine — **RTX 3050 Laptop, 6 GB VRAM, 34 GB RAM, NVMe, Windows 11
(WDDM), AC power**:

| seq (LM path) | step | tokens/s | peak VRAM |
|---:|---:|---:|---:|
| 128 | 10.70 s | 12.0 | 3.18 GB |
| 512 | 17.95 s | 28.5 | 3.82 GB |
| **1024** | **27.91 s** | **36.7** | **4.61 GB** |
| 2048 | 64.53 s | 31.7 | 6.60–7.93 GB (spills) |

The 32B weights are **19.2 GB**. They never fully live in VRAM. At 1024 tokens/step the
throughput is **3.1×** the 128-token rate, because the per-step I/O is amortised over
more tokens. 50 steps of Dolly-15k SFT moved eval loss **1.8755 → 0.9659**.

**Installed from PyPI? Start with [`QUICKSTART.md`](QUICKSTART.md) (T?rk?e:
[`QUICKSTART.tr.md`](QUICKSTART.tr.md)) ? or run `install.ps1` / `install.sh`, which
checks Python, installs the package, looks for a GPU and runs `doctor`.** — it uses only the
published CLI (no repo scripts). Deeper dive: [`GUIDE_6GB_32B.md`](GUIDE_6GB_32B.md) — the practical write-up: presets,
the six traps, and which numbers are real. The measurement protocol, the discarded rows
and the correction of an earlier wrong claim ("8.6× / 100 tok/s") are in
[`LONGCTX_FINDINGS.md`](LONGCTX_FINDINGS.md).

## Quick start (installed package, nothing else needed)

```bash
pip install streamft
streamft doctor                    # torch / CUDA / bitsandbytes / free VRAM

# will it fit? memory arithmetic from config.json + the measured reference
streamft fit --model Qwen/Qwen2.5-14B-Instruct --gpu 6 --seq 512

# one NF4 checkpoint per model (HF id or local directory)
streamft prepare --model Qwen/Qwen2.5-0.5B --save-quant checkpoints/qwen0.5b_nf4

# SFT on Dolly-15k — no data prep
streamft train --model-dir checkpoints/qwen0.5b_nf4 --sft --steps 20 --seq 512 --tag first

# LM path (optional): build token shards, then train without --sft
streamft data --model Qwen/Qwen2.5-0.5B --out data
streamft train --model-dir checkpoints/qwen0.5b_nf4 --steps 20 --seq 256 --tag lm

# no GPU at all, or no dataset: both are covered
streamft cpu --model Qwen/Qwen2.5-0.5B --steps 50 --seq 128
streamft synth --model Qwen/Qwen2.5-1.5B-Instruct --n 200 --out data/synth.jsonl

# inference budget (weights + KV cache), not just training
streamft fit --model Qwen/Qwen2.5-7B-Instruct --gpu 6 --seq 2048 --mode inference

# does it actually work? 20 local prompts, no API and no judge model
streamft eval --model Qwen/Qwen2.5-1.5B-Instruct
```

Every run appends one JSON row to `results/results.jsonl` next to where you launched it.
Six worked recipes: [`COOKBOOK.md`](COOKBOOK.md).
The same commands are in [`QUICKSTART.md`](QUICKSTART.md), which is what a pip user should
read — this README also documents the experiments behind the numbers.

### Spiking LM (from scratch)

A separate, small product line: LIF neurons, synaptic currents and a runnable
HTTP server. It is a research model, not a general assistant.

```bash
streamft spiking tokenize --text stories.txt --out data/stories.pt
streamft spiking train --data data/ts_train.pt --max-tokens 5000000 \
  --valid data/ts_valid.pt --out checkpoints/spiking_4m \
  --steps 300 --dim 256 --layers 1 --adapt 0.1
streamft spiking generate --model checkpoints/spiking_4m --prompt "Once upon a time"
streamft spiking serve --model checkpoints/spiking_4m --port 8011
streamft spiking ingest --model checkpoints/spiking_4m_10k \
  --text new_data.txt --old-data data/ts_train.pt --old-valid data/ts_valid.pt \
  --out checkpoints/ingest_demo --replay-fraction 0.2
streamft spiking ingest-watch --inbox data/inbox \
  --model checkpoints/spiking_4m_10k --old-data data/ts_train.pt \
  --old-valid data/ts_valid.pt --out checkpoints/watch --once
streamft spiking grow --model checkpoints/spiking_4m_10k \
  --out checkpoints/spiking_4m_grown_21m --dim 1152
```

It supports checkpointed resume sessions, safe incremental ingest with replay
and an accept gate, an inbox watcher that promotes only accepted versions,
truth-gated knowledge, function-preserving width growth, a `/generate` endpoint,
an OpenAI-shaped `/v1/chat/completions` endpoint and optional online
per-synapse rollback. See
[`docs/SPIKING_PRODUCT.md`](docs/SPIKING_PRODUCT.md),
[`docs/SPIKING_INGEST.md`](docs/SPIKING_INGEST.md),
[`docs/SPIKING_WATCH.md`](docs/SPIKING_WATCH.md),
[`docs/SPIKING_KNOWLEDGE.md`](docs/SPIKING_KNOWLEDGE.md),
[`docs/SPIKING_GROW.md`](docs/SPIKING_GROW.md),
[`docs/SPIKING_50M.md`](docs/SPIKING_50M.md) and
[`research/SPIKING_SHIFT.md`](research/SPIKING_SHIFT.md).

### Parked experiment: Qwen verified assistant

**Not the active direction.** The project direction is the from-scratch spiking
research line. This Qwen + retrieval experiment is kept as a parked product
prototype, not as the target model.

```bash
streamft assistant ask \
  --model models/Qwen2.5-1.5B-Instruct \
  --db data/knowledge.db \
  --question "What is the speed of light?"
streamft assistant serve \
  --model models/Qwen2.5-1.5B-Instruct \
  --db data/knowledge.db --port 8014
```

The server exposes `/health`, `/metrics`, `/v1/models`,
`/v1/chat/completions` and `/knowledge/search`. Answers carry source citations
from verified documents only. A 10-question held-out smoke eval scored 1.00
answer accuracy, 1.00 citation rate and 1.00 strict accuracy; the local server
runs at `http://127.0.0.1:8014`. See
[`docs/LOCAL_ASSISTANT.md`](docs/LOCAL_ASSISTANT.md).

**From a source checkout only** (not part of the wheel): `tools/finetune.py` is a wrapper
that encodes the VRAM presets, `scripts/compare_peft.py` reproduces the engine comparison,
`scripts/verify_gradients.py` the gradient check, `scripts/e2e_wheel.py` the clean-venv
install test (build ? fresh venv ? prepare ? train ? save adapter), `scripts/prep_data.py` the token shards
(the installed equivalent is `streamft data`), `scripts/hardware_report.py` prints a
machine + `doctor` + `fit` JSON row for an independent measurement, and `pytest tests -q`
the CPU suite.

`tools/finetune.py` encodes the VRAM presets we measured, because the *combination* of
flags decides whether a run fits — not any single flag:

| model | resident blocks | shared tensors | hard cap |
|---|---:|---|---:|
| 32B, ≤2k seq | 2 | `as-stored` | 5.3 GB |
| 32B, >2k seq | 1 | `as-stored` | 5.3 GB |
| 14B | 6 | `as-stored` | 5.2 GB |
| 7B / 3B / 1.5B | autotune | — | — |

## How it works

```
disk (NF4 safetensors)
  └─ mmap ──► one decoder block at a time ──► GPU ──► forward ──► drop
                                                    │
                                            recompute in backward
```

- `streamft/disk_quant_store.py` — mmap'd checkpoint, block loader, LRU layer cache
  (GPU or host RAM for the recompute inputs).
- `streamft/stream_quant_lora.py` — the streaming trainer: base frozen, LoRA trainable,
  loss chunked over the sequence so activations stay small.
- `streamft/stream_train.py` — per-layer forward hooks, chunked loss/backward,
  streaming eval CE.
- `streamft/autotune.py` — chooses the resident-block window from measured VRAM/RAM.
- `streamft/data.py`, `streamft/metrics.py`, `streamft/lora.py` — samplers, append-only
  JSONL log, minimal LoRA.
- `streamft/cli.py` — the engine CLI (`--prepare`, SFT/LM modes, `--eval_seq`,
  `--activation-cache cpu`, `--head nf4`).

The training tag in the evidence log is still `exp54_disk_qlora`, so rows written by
this engine and by the original experiment are directly comparable.

## Older GPUs (GTX 10/16-series, 4–8 GB)

Pre-Ampere cards (GTX 1060/1660, Pascal/Turing) have **no bf16 units**. Use fp16:

```bash
streamft train --model-dir checkpoints/qwen7b_nf4 --sft --steps 1000 \
  --seq 512 --blocks-on-gpu 1 --compute-dtype fp16 --vram-hard-gb 5.3
```

- On the reference Ampere card fp16 is slightly faster than bf16 at 7B (1.79 vs 1.91 s/step,
  same 5.36 GB peak); on Pascal it is the only option.
- On a **4 GB** card use `--blocks-on-gpu 1`, `--seq 512` and `--vram-hard-gb 3.4`.
  Those rows are not measured yet — if you run one, open a
  [hardware report](.github/ISSUE_TEMPLATE/hardware-report.md) and it can become a measured
  entry in `streamft fit`.
- `streamft fit` prints this hint automatically when it sees a pre-Ampere GPU.

### Tokenizer cost and the diffusion question

`seq` is in tokens, and tokenizers are not language-neutral. With the Qwen2.5 tokenizer,
the same passage costs **1.13 tokens/word in English, 1.74 in German and 2.39 in Turkish**
? a Turkish run at `seq 512` sees about half the text an English run does. Measured with
`scripts/tokenizer_cost.py`; the numbers, the byte-level alternative and why diffusion LMs
(LLaDA/Mercury: 800?1,500 tok/s at batch 1, but the same memory footprint) do not change
this project's bottleneck are in [`docs/TOKENIZERS.md`](docs/TOKENIZERS.md).

### Run it in a browser (no server)

```bash
streamft merge --model Qwen/Qwen2.5-0.5B --adapter checkpoints/cpu_cookbook --out merged/q05
powershell -ExecutionPolicy Bypass -File scripts/export_gguf.ps1   # -> q05-q4_k_m.gguf
python scripts/serve_static.py --port 8765
# open http://127.0.0.1:8765/wasm/index.html?model=../q05-q4_k_m.gguf
```

Measured on the reference laptop, 0.5B Q4_K_M (379 MB): **1.3 tok/s** with the plain
`http.server`, **6.0 tok/s** behind `scripts/serve_static.py` ? the COOP/COEP headers are
what let wllama use 12 threads instead of one. Details and limits in
[`wasm/README.md`](wasm/README.md).

### Share one machine on the LAN

One PC with the model can serve everyone on the same network:

```bash
streamft serve --model merged/q05_cpu --port 8000
# others open http://<that-machine-ip>:8000
```

It ships a small chat page and an OpenAI-compatible `POST /v1/chat/completions`, so existing
tools can point at it. This is **one machine computing, the rest as clients** ? not
peer-to-peer sharding and not volunteer compute. Tested end to end: `/health` and a
16-token completion (1.9 s on the reference GPU).

### Share one base, ship 5 MB adapters

A LoRA adapter is the per-behaviour artifact. Pack it to 4 bits and serve it on top of a
shared base:

```bash
streamft adapter --adapter checkpoints/my_adapter.pt --bits 4 --out checkpoints/my_adapter_q4.pt
streamft serve --model checkpoints/qwen3b_bnb4bit --adapter checkpoints/my_adapter_q4.pt
```

Measured on the reference machine (Qwen2.5-3B, 60-step Dolly LoRA):

| storage | size | bits / base-param |
|---|---:|---:|
| fp32 | 29.6 MB | 0.0766 |
| fp16 | 14.8 MB | 0.0382 |
| 8-bit | 8.56 MB | 0.0222 |
| **4-bit** | **4.88 MB** | **0.0126** |

**What the bit number means.** The base is 4-bit NF4 and the adapter is stored 4/8-bit.
`0.0126 bit/base-param` is the **marginal adapter cost**, not a 0.1-bit base. Count the base
and it is 4 bit/param (4-bit base) or 16 (bf16). The 0.1-bit VQ model was a failed
experiment; the sub-0.1 artifacts are adapters, prompts and coresets on a shared base.

**FAQ: “So are you using 0.1-bit?”** No. The deployed precision is **4-bit NF4** for the
base and 4/8-bit for the adapter. `0.0126` is an accounting ratio with the base shared. The
130 KB and 1.7 MB demos are separate artifacts (data and a linear model), not quantized
LLMs. There is no 0.1-bit general model here.

870 held-out Dolly rows, masked SFT loss: base 2.3752, original 2.3403, 8-bit 2.3401
(p = 0.81, no measurable loss), 4-bit 2.3420 (+0.0018, p = 0.001, 5% of the gain). A
second disjoint seed (pure LoRA, frozen base) gives 8-bit +0.00108 (p = 0.038) and 4-bit
+0.00114 (p = 0.029): a ~0.001 cost at both widths. The cost is small and real; 8-bit is
the conservative default. A second task, SST-2 (2,000 train / 400 test), goes from
**71.25%** base to **93.75%**, and
both packed adapters stay at **93.75%** (masked loss 0.7940/0.7942 vs 0.7930; p = 0.50 /
0.41). A third task, AG News (4-way), goes from 2.50% with the bare prompt (not followed)
to **62.25%**; 8-bit and 4-bit stay at **62.00%**. The adapter loads in 0.115 s. The LoRA
path is ~26% slower than the base on this card (8.75 vs 11.79 tok/s; a bf16 LoRA path
reaches 9.13); merge it into a bf16 base for zero per-token overhead, or accept the LoRA
path when storage/switching matters more than throughput.
Rank reduction is not viable: SVD to rank 8 loses 35.8% of the update.

**A prompt can be the better rung.** On the same base, a 441-byte 4-shot SST-2 prompt
scores 94.0% (adapter 93.75%) and a 242-byte 4-shot AG News prompt scores 61.75% (adapter
62.25%). If a few examples express the task, use the prompt; the adapter is for behaviour
a prompt cannot express, or when prompt tokens/context cost more than the storage.

**4-bit is a memory win, not a speed win.** On Qwen2.5-0.5B (bf16 fits on the card), bf16
generated **16.8 tok/s** vs **13.1** for 4-bit NF4 (~22% slower), while peak VRAM fell from
**1.00 GB to 0.49 GB**. The bit width is how a larger model fits; it does not make the
math faster.

### Measured on

Every speed number here comes from **one machine**: RTX 3050 Laptop 6 GB, Windows 11,
AC power, NF4 base, LoRA r=8, batch 1, Dolly-15k SFT. The four rows (3B/7B/14B/32B) are
listed in [`docs/MEASURED.md`](docs/MEASURED.md), together with the rules for adding one
from a different card:

```bash
python scripts/measured_row.py --row "<json line>" --config checkpoints/<model>
```

It validates the run (finished steps, `grad_layers == n_layers`, peak VRAM recorded),
derives the architecture fingerprint and prints the snippets for `fit.py`, `fitcore.py`
and the Space. Anything not in that table prints **"not measured"** in `streamft fit` ?
that is the point.

### Why 0.1-bit compression is not a universal target

A measured note on quantization vs distillation: post-hoc quantization dies below 3 bits;
a 340 KB student matches a 1.5B teacher on a narrow keyword task at 0.0019
bit/teacher-param; the same 4.4M student is within 1.3 points on DBPedia-14 but loses
22.5 on SST-2 and 47.9 on MNLI; open-ended generation dies at 3 bits. A cheap 45 KB TF-IDF
student is a **lower bound** on the 4.4M student's normalized skill (6 tasks, R² = 0.89),
not a predictor: QQP (C 0.21, S 0.49) breaks the near-identity. Details, caveats and
reproduction: [`docs/BEHAVIORAL_BITS.md`](docs/BEHAVIORAL_BITS.md).

## Verification

`python scripts/verify_gradients.py --model-dir checkpoints/qwen3b_bnb4bit --seq 128`

- **One resident block vs all 36 resident** (identical adapter weights, Qwen2.5-3B NF4,
  seq 128): the loss is identical to 10 decimals and the adapter gradients are
  **bit-identical** — 288 tensors, max relative error `0.0`, global relative error `0.0`.
  This is the streaming/recompute claim: dropping blocks and recomputing them in the
  backward changes nothing.
- **Manual block backward vs PyTorch autograd** through the same forward: global relative
  error **1.33%** (cosine `0.9998`). The same 1.33% appears with *all* blocks resident,
  so it is bf16 dequantisation precision, not the streaming path.
- Finite differences cannot check this on a 4-bit base: the LoRA delta is added to a bf16
  tensor, so a one-entry perturbation is below one bf16 ULP (measured for the same entry:
  `eps=2e-3` → `-0.12`, `eps=2e-2` → `+0.02`). The chunked-loss gradient itself is
  checked exactly in `tests/test_streamft_smoke.py`.

Raw output: `results/gradcheck.json` (git-ignored; the numbers above are copied from it).

## Compared with PEFT (matched runs)

Same data, same LoRA (r=8, alpha=16, q/k/v/o), same AdamW lr 1e-4, batch 1, same prompt
masking, same token-weighted masked eval on 8 held-out rows. PEFT uses
`prepare_model_for_kbit_training` with gradient checkpointing (the standard QLoRA setup).
Step times move by up to ~30% between runs on this laptop; quality is the stable number.

**3B (fits entirely):**

| | streamft (36 blocks resident) | PEFT / transformers |
|---|---:|---:|
| step time, median of 30 @ seq 512 | 0.94–1.24 s | **0.58–0.88 s** |
| peak VRAM | **3.85 GB** | 4.90 GB |
| eval before | 1.42087 | 1.42087 |
| eval after | **1.30720** (−0.1137) | 1.33002 (−0.0909) |

**7B / 14B / 32B (do not fit comfortably), after the masked-gradient fix:**

| model | step time, median @ seq 512 | peak VRAM | eval (start → after, 30 steps) |
|---|---:|---:|---:|
| 7B, streamft (28 blocks) | **1.55 s** | **5.51 GB** | 2.1928 → **1.1055** |
| 7B, PEFT | 8.96 s | 10.05 GB | 2.1914 → 1.1491 |
| 14B, streamft (6 blocks) | **5.86 s** | **3.86 GB** | 2.3748 → **0.9742** |
| 14B, PEFT | 18.56 s | 16.48 GB | 3 steps only |
| 32B, streamft (2 blocks, seq 1024) | **12.48 s** | **3.90 GB** | 1.8755 → **1.0008** |
| 32B, PEFT | fails before step 1 | — | — |

At 7B the crossover has already happened: PEFT's 4.4 GB of weights plus activations no
longer fit, so it spills (10.05 GB reserved) and pays ~5.8× the step time. Note that
streamft's 7B peak (5.51 GB) is *higher* than its 14B peak (3.86 GB) because the 7B run
kept all 28 blocks resident while the 14B run kept 6 — the window is a choice, and the
14B number is the one to copy for a 6 GB card.

**32B: PEFT does not reach the first step.** The 19.2 GB of weights load only by spilling
into shared memory (19.45 GB reserved on a 6 GiB card), and `prepare_model_for_kbit_training`
then dies allocating the fp32 norm copy: *CUDA out of memory. Tried to allocate 2.90 GiB;
18.00 GiB is already allocated.* The streaming engine runs the same checkpoint at
**12.5 s/step and 3.90 GB** (64/64 layers).

The 3B row is the honest answer to "is this faster?": **no** — on a model that fits, PEFT
is faster and uses 1.05 GB more. The 14B/32B rows are the design point: PEFT does not
OOM on Windows, it **spills into shared system memory** and pays 2.8–5.8× the step time.

### Correction: masked gradient (25 Oct, after an external review)

`chunked_lm_loss_and_backward_masked` divided every chunk by **that chunk's** valid-token
count. The reported loss was still the correct mean, so the curves looked right, but the
accumulated gradient over-weighted chunks with few unmasked tokens — i.e. the prompt-heavy
start of an SFT row. Measured on a toy model with an uneven mask: **112–427% relative
gradient error** at seq 512/1024 with 60% of the row masked (chunk 128), while balanced
masks were unaffected.

The fix divides every chunk by the **global** valid count (computed once, no-grad), so the
gradient is exactly d(mean CE over valid tokens)/dh. Two tests pin it:
`test_masked_chunked_gradient_matches_full_with_uneven_masks` and
`test_masked_chunked_handles_fully_masked_chunks`.

All SFT runs above were re-run with the fix. Quality held: 3B −0.114 (was −0.119),
7B −1.087 (was −1.083), 32B −0.875 (was −0.880); 14B is now 30 steps (−1.40) instead of
10. The numbers in the tables are the post-fix ones.

Reproduce: `python scripts/compare_peft.py --model checkpoints/qwen3b_bnb4bit --steps 30`
and `... --model checkpoints/qwen14b_nf4 --steps 30`.

### Reproducibility probe

Run this and compare the row; it is deterministic (seed 1234) apart from GPU-kernel
non-determinism:

```bash
streamft train --model-dir checkpoints/qwen3b_bnb4bit --steps 3 --seq 128 \
  --eval_chunks 1 --tag repro
```

Reference on the only machine tested so far (RTX 3050 Laptop 6 GB, torch 2.13.0+cu126,
bitsandbytes 0.50.2, Windows 11):

```text
losses        3.347981  3.647039  3.049560
ce_before     2.44565
ce_after      2.44568
peak_gb_max   3.068
adapter_sha_after  422424c59ef004c5
```

Different hardware will differ in the last digits (kernels, bf16 rounding). What must
match is the shape: the same three losses to ~1e-3, `grad_layers == n_layers`, and a
changed adapter hash. If it does not, that is a bug report worth posting with the
`streamft doctor` output.

## What this does not do

- **2k context does not fit** on 6 GB. It is ~0.6 GB short: with `--activation-cache cpu`
  the 2k peak is 6.60 GB, and what remains is the bf16 `lm_head` (1.56 GB). The 4-bit
  `--head nf4` path is **broken in the bitsandbytes version here**.
- **Longer context is not a quality lever by itself.** At a matched token budget
  (122,880 tokens), training at seq 512 gave a 2k-context CE of 1.2833; training at
  seq 2048 gave 1.3033. Data volume is the lever.
- **SFT `--seq` is a truncation cap**, not a padding target: one dataset row per step.
  Dolly rows are 43–1964 tokens, median 134. Never compute tokens/s from `--seq` on
  that path.
- Speed numbers are medians of warm runs, AC power only. Battery/Modern-Standby runs are
  invalid and were discarded.
- Single GPU, Windows/WDDM. Linux should work (no WDDM shared-memory cliff) but is not
  measured here.

## Repository layout

```
streamft/            the engine (package)
tools/finetune.py    one-command wrapper with the measured presets
scripts/             longctx_sweep.py — the LM-path measurement sweep
GUIDE_6GB_32B.md     practical guide
LONGCTX_FINDINGS.md  protocol, curve, the 2k wall, the E4 result, corrections
LONGCTX_PLAN.md      the measurement plan
ARTICLE.md           the DEV Community post source
results/             the engine appends results.jsonl here (git-ignored)
```

## The research line

This repository grew out of an attempt to replace dense attention with a **learned
block-sparse router**. That line is mostly a negative result and now lives on its own, as
an archive: **[byjameson/lcga-local](https://github.com/byjameson/lcga-local)** —
[`LCGA_FINDINGS.md`](https://github.com/byjameson/lcga-local/blob/main/LCGA_FINDINGS.md),
all experiments, every raw JSON, and the three measurement errors that were corrected.
The short version: on plain language modelling the learned router ties a budget-matched
static window; on retrieval data it reaches dense exact match at a 25% block budget.

History: this repository was previously named `lcga-local`; GitHub redirects the old
URLs. The engine lived at `experiments/exp54_disk_qlora.py` + `lcsa/` until the split.

## Articles

- [Fine-tuning a 32B model on a 6 GB laptop GPU — and the three measurements that nearly fooled me](https://dev.to/mycute_97b39c4e34027570f9/fine-tuning-a-32b-model-on-a-6-gb-laptop-gpu-and-the-three-measurements-that-nearly-fooled-me-2ojm)
- [Running a 4B model in 6 GB and training 3B parameters on a laptop](https://dev.to/mycute_97b39c4e34027570f9/running-a-4b-model-in-6-gb-and-training-3b-parameters-on-a-laptop-4al)
- [The same 32B post on Medium](https://medium.com/@myredditacc9393/fine-tuning-a-32b-model-on-a-6-gb-laptop-gpu-e749df447229)
- [Will it fit? A free planner for fine-tuning models that do not fit](https://dev.to/mycute_97b39c4e34027570f9/will-it-fit-a-free-planner-for-fine-tuning-models-that-do-not-fit-on-your-gpu-o50)
- [Will it fit? on Medium](https://medium.com/@myredditacc9393/will-it-fit-a-free-check-before-you-download-19-gb-of-weights-9097e1fcf93f)

### Behavioral and watermark research

- [Behavioral vaccination: a 400-row mixed prime protects a behavior](https://dev.to/mycute_97b39c4e34027570f9/behavioral-vaccination-a-400-row-mixed-prime-protects-a-behavior-25ac)
- [4-bit Naive Bayes: symmetric fails on AG News, zero-point rescues it](https://dev.to/mycute_97b39c4e34027570f9/4-bit-naive-bayes-works-on-sms-fails-on-ag-news-4p62)
- [Behavioral epistasis: a watermark that works alone dies in every merge](https://dev.to/mycute_97b39c4e34027570f9/behavioral-epistasis-a-watermark-that-works-alone-dies-in-every-merge-1j9f)
- [The watermark's logit rises smoothly; its generation falls off a cliff](https://dev.to/mycute_97b39c4e34027570f9/the-watermarks-logit-rises-smoothly-its-generation-falls-off-a-cliff-4j0c)
- [Joint training did not fix the watermark merge: 8/8 alone, 0/8 at 0.75](https://dev.to/mycute_97b39c4e34027570f9/joint-training-did-not-fix-the-watermark-merge-88-alone-08-at-075-1hka)

**Spiking prototype:** a 4.3M-parameter LIF language model with synaptic current dynamics
learns TinyStories at 2.05 bpb after 300 steps and 1.94 after 1000 steps on the 6 GB card;
adding spike-frequency adaptation reaches 1.84. The rate control is weaker and saturates.
This is the from-scratch starting point for the brain-like line, not a claim that spiking
beats transformers. [`research/SPIKING_LM.md`](research/SPIKING_LM.md)

**Spiking rollback:** after a TinyStories -> WikiText shift, restoring the top 10% of
phase-A synapses by Fisher importance recovers 95.0%/94.1% of the old-task performance
while keeping 18.3% of the new-task gain on two seeds. Global rollback restores the old
task but discards all new learning. [`research/SPIKING_SHIFT.md`](research/SPIKING_SHIFT.md)

**Scaling check:** a 20.97M model trained for 10,000 steps / 164M tokens reached
1.327 bpb; the 4.3M model reached 1.336 on the same budget, and the larger model was
slightly worse on a fixed sample-quality probe. More parameters did not buy quality at
this budget; the next attempt needs 500M+ tokens. [`docs/SPIKING_PRODUCT.md`](docs/SPIKING_PRODUCT.md)

**Generative data diversity (first generative result):** a 4.3M model trained for
3,000 steps on 16M tokens. TinyStories-only: TinyStories **1.566** / WikiText
**4.455** bpb. Mixed 25% WikiText (12M TinyStories + 4M WikiText): TinyStories
**1.605** / WikiText **2.927** bpb. Longer mixed training (10k steps, 40M
tokens) improves both: TinyStories **1.464** / WikiText **2.794** bpb, versus
1.302/4.003 for the TinyStories-only 10k model. 50% and 75% WikiText add little
WikiText gain at a larger TinyStories cost (50%: 1.702/2.906; 75%:
1.734/2.820). The 21M/50M TinyStories-only models stay at WikiText 4.03, so
data diversity moves the general-text metric where parameter scaling did not.
The 21M 10k mix at matched 41M tokens (batch 32) is still worse on all three
bpb (1.669 / 3.011 / 3.384 vs 1.464 / 2.794 / 3.249); 5x parameters do not buy
quality at this budget. The batch-16 run (20.5M tokens) is worse still
(1.774/3.108). On a held-out
world-facts article set, the 4.3M mix reaches **3.249** bpb versus **3.949**
for TinyStories-only 10k, so the diversity gain transfers beyond WikiText-2.
A TinyStories-first curriculum does not dominate: it is slightly better on
TinyStories (1.439) but worse on WikiText (2.964) and world facts (3.374);
pure mix remains the best general-text tradeoff. Generation samples remain
locally repetitive and ungrammatical; 21M/50M TinyStories-only samples are more
story-shaped but still repetitive. Tighter sampling helps the TinyStories-only
4.3M model but hurts the mix model, so the generation gap is not just a
temperature problem. Lower bpb is not yet a generation-quality or deployment
win.
[`research/SPIKING_GENERATIVE.md`](research/SPIKING_GENERATIVE.md)

**Tiny Spiking Lab (product path, release blocked):** the local ingest workflow
improves a 76-article held-out domain from **5.99 to 4.45 bpb** in 300 steps
with 20% replay. It accepted the update on a 4096-token eval (old +2.55%), but
the full 8192-token eval shows old **+3.29%** and unmonitored WikiText **+7.1%**;
generation is formatting noise or corrupted text. A naive story ingest adapts
the target (space-story 2.19 -> 1.66, robot-themed samples) but full eval shows
TinyStories **+13.6%** and WikiText **+33.9%**. A safer recipe (70% replay from
a 20M TinyStories + 5M WikiText old mix, LR 5e-5, 120 steps) gives target
**2.19 -> 0.72**, TinyStories **+4.1%**, WikiText **-13.4%**. A second corpus
(Nova) replicates it: target **2.05 -> 0.60**, TinyStories **+4.4%**, WikiText
**-13.6%**. A second seed per corpus also replicates (TinyStories +4.1-4.4%,
WikiText -13.2-13.6%). The **narrow story-lab gate** passes on two synthetic
corpora and two seeds. **The real public-domain cross-book test fails**:
Alice-train -> Oz-valid passes bpb (target **-11.7%**, TinyStories **+3.5%**,
WikiText **-14.0%**) but generation alpha collapses on `Dorothy opened the
door` (min **0.534** seed 0, **0.259** seed 1, versus base 0.727-0.751). Gate
**BLOCKED**; the synthetic pass does not generalize to real text. A small-model
efficiency probe: 4.3M runs at **1,171 tok/s** on CUDA and **730 tok/s** on CPU
with **24.8 MB** peak VRAM; 21M runs at 563/204 tok/s with 88.3 MB. This is not
a general model; packaging/licence/model card are still missing.
[`research/SPIKING_TINY_LAB.md`](research/SPIKING_TINY_LAB.md)

**Tiny Transformer Chat (product path):** a **20.2M from-scratch transformer**
on the same clean chat data reaches chat bpb **1.359**, assistant loss **4.01**,
alpha min **0.731** and 0 empty samples. With role tokens banned at decoding it
passes the smoke gate; the 21M/50M spiking chat models remain BLOCKED at
2.29-2.31 bpb / 6.7-6.8 loss. A 49.2M transformer adds only 0.015 bpb, so the
20M checkpoint is the product model. It is a tiny, repetitive character model,
not a general assistant. Model card:
[`docs/TINY_TRANSFORMER_CHAT.md`](docs/TINY_TRANSFORMER_CHAT.md); local server:
`python -m streamft.chat_server`; demo: `scripts/chat_demo.ps1`.

**Sleep-like replay:** during a TinyStories -> WikiText phase, mixing 20% old windows
into every batch keeps old bpb at **2.079** (phase-A 2.077; naive 2.632) and retains
**99.2%** of the new-task gain. Replay is much better than post-hoc shadow rollback
here. [`research/SPIKING_REPLAY.md`](research/SPIKING_REPLAY.md)

**Replay + rollback:** controlled seed-1 arms confirm the mechanisms are not
additive. Replay alone keeps **98.9%** of the new gain; replay followed by Fisher
rollback keeps only **16.0%**. Replay prevents the forgetting; rollback after it
spends the new learning. [`research/SPIKING_REPLAY_ROLLBACK.md`](research/SPIKING_REPLAY_ROLLBACK.md)

**21M, two seeds:** replay-first holds at 21M. Replay keeps **99.6-99.8%** of the
new-task gain; replay+rollback keeps **6.5-12.0%**; replay+EWC keeps **99.3%**.
EWC is safe but does not beat simple replay.
[`research/SPIKING_REPLAY_EWC_21M.md`](research/SPIKING_REPLAY_EWC_21M.md)

**600-step phase B + generation probe:** at 21M and two seeds, replay keeps
**99.7%** of the new gain; naive samples collapse from **144-151 to 78-81 chars**
and repeated 4-grams quadruple. Replay keeps the phase-A profile; EWC is slightly
better on distinct-2/length but costs new-task bpb.

**50M long phase B:** replay keeps **98.2%** of the new gain and improves the
held-out continuation probe (content F1 **0.140 -> 0.182**); EWC keeps 91.7% and
lowers it to 0.115; naive drops it to 0.000. This is a small semantic signal, not
semantic competence. [`research/SPIKING_LONG_REPLAY.md`](research/SPIKING_LONG_REPLAY.md)

**Narrative-coherence benchmark:** 100 held-out four-way next-sentence cases —
4.3M **0.29**, 21M **0.41**, 50M **0.40** (chance 0.25). 50M does not beat 21M,
and EWC lambda 0.5/2.0 stays at 0.39/0.40 while costing new-task bpb. Parameter
scaling is not buying semantics. The decision is to revise architecture/data,
not just scale. [`research/SPIKING_SEMANTIC.md`](research/SPIKING_SEMANTIC.md)

**Top-down feedback (negative):** adding a zero-initialized last-layer-to-input
feedback path to the 21M model scored **0.39** narrative accuracy vs **0.45** for
the baseline after the same 500 steps, and slightly worse bpb. This topology did
not help. [`research/SPIKING_FEEDBACK.md`](research/SPIKING_FEEDBACK.md)

**Variable binding (non-TinyStories, negative):** on tiny three-statement stories
with **disjoint** train/test (person, color, object) triples, the 128-wide spiking
model reaches **0.168** (below the 0.25 chance) while the rate control reaches
**0.318**. Both memorize training triples and fail held-out binding; the current
architecture needs a memory/attention mechanism, not more parameters.
[`research/SPIKING_BINDING.md`](research/SPIKING_BINDING.md)

**Binding benchmark audit:** the first task is solvable **78%** by matching color
alone and **76%** by object alone, so a corrected conjunction split was built
(both attributes required, unique pair). There spike reaches **0.220**, rate
**0.326**, while a 2-layer transformer with dropout reaches **0.886** and a
4-layer transformer **1.000**. At 256 wide, spike/rate improve to **0.519/0.629**
on average over two seeds (seed range **0.466-0.572** / **0.502-0.756**) but are
**position-biased**: the spike model is best on the third statement (0.846/0.872)
and weak on the second (0.377/0.182). A 512-wide run is worse (**0.446/0.406**),
so width alone does not close the gap. A simple attention readout hurts
(**0.364/0.384**); a fixed pointer (**0.236/0.514**) and dropout
(**0.308/0.460**) also fail.

**Statement memory (positive):** a 256-wide spiking model with a statement-level
name/attribute memory plus a copy loss and a **self-supervised matching
attention** loss reaches **0.996/1.000** across two seeds on held-out
conjunction binding, is balanced across all three positions, and is
permutation-invariant (**0.996/1.000** consistency). The attention target is
derived from the query/statement attributes, not the answer. Without it, 6k
steps give **0.308/0.696**; at 12k the rate control reaches **0.874** but the
spike model is **0.470** and strongly position-biased (middle statement 0.159).
The architecture can bind; unsupervised credit assignment is still open.
[`research/SPIKING_BINDING_CONJ.md`](research/SPIKING_BINDING_CONJ.md)

**Continual binding (two seeds + hard split):** naive phase B on 16 new triples
drops old accuracy from **0.974/1.000** to **0.322/0.544** and held accuracy from
**0.730/0.926** to **0.240/0.280**. A 20% replay mix keeps old at
**0.992-0.996**, new at **1.000** and held at **0.788-0.866**. On a harder color
shift (old red/blue -> new green/yellow), naive old falls **0.930/1.000 ->
0.700/0.702** and held **0.336/0.000 -> 0.478/0.662**; replay reaches
**1.000/1.000/0.974** and **1.000/1.000/0.998**. Rollback 10% alone is weak
(0.392/0.674 old on the triple split) or destroys new colors (new 0.366 on color
seed 2); replay+rollback and replay+EWC are close but not consistently better.
Replay is the most reliable arm.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Without the matching-attention loss on harder splits (negative):** copy-only
color phase A is **0.332/0.214** (seed 0/2, at or below chance), and a no-overlap
2-name split collapses (**0.474/0.644**, even with the attention control). The
attention loss remains a required inductive bias at this budget; replay only
helps once phase A is learnable.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Four-statement stories:** dynamic memory, per-statement slots, slot reset, and
a hard name slot all stay at **0.214-0.236** old accuracy (majority **0.26**)
after 5k steps. Changing the slot key to the statement's **color+object
embeddings** and using **4-head attention** solves phase A: old **1.000**, all
four positions **1.000**, old permutation **1.000**. Copy-only phase B (no
matching-attention loss) keeps old **0.996/0.970**, new **1.000**, held
**0.872/0.728**, permutation **0.996/0.972**; replay reaches
**1.000/1.000** and held **0.926/0.760**. The structured key is the mechanism;
the task-specific attention loss is unnecessary and can hurt retention (naive
old **0.220/0.444** with it).
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Five-statement copy-only (positive):** five names, five attr-key slots,
sequence length 32. Phase A old **0.998** with all positions balanced and
permutation **1.000**. Copy-only phase B keeps old **0.950/0.994** (naive) or
**1.000** (replay), new **1.000**, and old permutation **0.944/0.994** or
**1.000**. Held-out composition is seed-sensitive (**0.244/0.356** on seed 0,
**0.982/0.980** on seed 1), so the next issue is composition, not retention.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Six-statement copy-only (positive, gate passed):** six statements, six names,
sequence length 37, six attr-key slots. Phase A old **1.000** with all positions
and permutation **1.000**. Copy-only phase B: naive old **0.998/0.914**, new
**1.000**, held **0.988/1.000**, permutation **0.998/0.924**; replay
**1.000/1.000**, held **1.000/0.998**, permutation **1.000/1.000**. Balanced
positions, permutation invariance, and held composition all pass on both seeds.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Seven-statement copy-only (positive, gate passed):** seven statements, seven
names, sequence length 42. Phase A old **1.000/0.998**, all positions and
permutation **1.000**. Copy-only phase B: naive old **1.000/0.986**, new
**1.000**, held **0.998/0.966**, permutation **1.000/0.986**; replay
**1.000/1.000**, held **0.988/0.994**, permutation **1.000/1.000**. The gate
passes on both seeds at seven statements.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Relaxed structure (partial):** variable filler lengths 0/1/2 with
token-triggered boundaries/read. Replay keeps old/new at **1.000** and
permutation at **1.000**, but held composition drops to **0.803/0.651**. A
0->1->2->mixed filler curriculum with 6k stories and 9k phase-A steps gives
**0.786/0.611** — no improvement. The gate fails on held-out composition, not on
old/new or balance; more filler-length practice does not fix it.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Factorized color-object attention (positive, gate passed):** separate color
and object keys/queries with multiplied attention distributions. Relaxed
structure: old/new/held **1.000/1.000/1.000**, permutation **1.000**, and held
**1.000** at every filler length, on both seeds. The earlier held gap was the
additive key, not data or training.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Eight-statement factorized relaxed (positive, gate passed):** eight names,
eight slots, sequence length 63, fillers 0/1/2. Replay: old/new/held
**1.000/1.000/1.000**, permutation **1.000**, held **1.000** at every filler
length on both seeds. The factorized mechanism scales to eight statements.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Multi-token answer (positive with curriculum):** answer is name + color, with
a query-color copy for the second token. Seed 0 reaches exact
**1.000/1.000/1.000** and permutation **1.000**; seed 1 initially fails the name
read (old **0.231**, new **0.354**, held **0.341**) though its color token is
**1.000**. A 3k red-only warmup + 6k red+blue + 3k phase B fixes both seeds to
exact **1.000/1.000/1.000**, permutation **1.000**, name/color 1.000 at every
filler length.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Learned boundaries, two-token answers (positive, gate passed):** statements
end with a generic `|`; learned write/read gates replace the known `.`/`?`
boundaries. Two seeds: phase-A old **1.000**, phase-B old/new/held
**1.000/1.000/1.000**, permutation **1.000**, name/color 1.000 at every filler
length. The remaining structure is a task-specific color/object factor and a
task-specific color-copy readout, not a general language model.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Raw boundaries, no separator (positive, gate passed):** statements are
concatenated with no separator token at all. A write candidate on the object
token plus a context read gate gives the same result: two seeds, phase-A old
**1.000**, phase-B old/new/held **1.000/1.000/1.000**, permutation **1.000**,
name/color 1.000 at fillers 0/1/2. The gate is supervised with object positions,
so this is learned write/read classification, not unsupervised segmentation.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Natural-language templates (positive, gate passed):** English-like
`tom has a big red ball.` / `who has a big red ball? mia red.` with learned
`.`/`?` gates. Two seeds: phase-A old **1.000**, phase-B old/new/held
**1.000/1.000/1.000** (seed 1 held **0.9989**), permutation **1.000**, all
filler cells above 0.95. Still a fixed 24-word template set, not free-form
language.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Wide natural-language templates (positive, gate passed):** verbs, articles
and adjectives vary (`tom owns the old red ball.` / `who found a big yellow
car? sam yellow.`). Two seeds: phase-A old **1.000**, phase-B old/new/held
**1.000/1.000/1.000**, permutation **1.000**, all 18 split/filler cells 1.000.
Still a fixed 31-word template set, not free-form language.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Paraphrase templates (scoped positive, 4 pairs):** names and queries move
(`tom has a red ball.` / `the red ball belongs to tom.` / `whose red ball is
it?`). Category states plus a template curriculum and 50% replay give two seeds
old/new/held **1.000/1.000/1.000**, permutation **1.000**. The full 16-pair set
is seed-sensitive and stays a negative result.
[`research/SPIKING_BINDING_CONTINUAL.md`](research/SPIKING_BINDING_CONTINUAL.md)

**Curriculum without the attention loss (negative):** a padded 1 -> 2 -> 3
statement curriculum trained with the copy loss only reaches spike **0.412**
(permuted 0.392, last-position-biased) and rate **0.212**. It does not replace
the self-supervised matching-attention loss at this budget.
[`research/SPIKING_BINDING_CONJ.md`](research/SPIKING_BINDING_CONJ.md)

## Support the work / Destek

This project is developed on a 6 GB laptop GPU. If you find it useful and want
to support further experiments, donations go toward more compute and larger
runs. There is no paid support, token, or investment offer. Canonical addresses
and network warnings: [`docs/DONATE.md`](docs/DONATE.md).

- **EVM-compatible networks** (Ethereum, BNB Smart Chain, Polygon, Optimism,
  Arbitrum One, Base, Avalanche C-Chain):
  `0xa4dEc84Fe6F13c8A3b8707446D6892a6E272285c`
- **Solana:**
  `EDqs8w4JkS5vAAfxmsB7fAQwoauEYufYXvMWFW9aYPdc`
- **Bitcoin:**
  `bc1qsq4mdtf0dlfvppllqwdaknxrq6xkjgsrx6ctgm`

Please double-check the network and the full address before sending; crypto
transfers are irreversible. Donations are voluntary and do not buy influence
over results or reviews.

## License

Apache-2.0 for code and model weights — see [`LICENSE`](LICENSE) and
[`NOTICE`](NOTICE). The notice lists the training-data sources and their
original licences; dataset text remains under those terms. Atom Zero is
open source; third-party hosting is the host's responsibility.
