Background, 17 Oct 2026; superseded on 18 Oct. The literature map and propositions here led to the decisions in docs/requirements.md. Where they disagree, the requirements win. What changed since this page:
No trained latent memory in v0. cogito-mem v0 is a frozen EmbeddingGemma 2 plus keyword search over the stored messages. The trained tape, the librarian (E36) and the latent notes on this page stay with the MrCogito research line.
No typed questions. The Jev-style protocol proposed here (value, holds, choice, when, facts) was dropped. Keys are always static, and a query returns stored messages only.
Who it is for. People's own chats with Claude, Codex and ChatGPT: getting back "that idea from a week ago" that nobody wrote down.
MCP and setup. MCP ships in v0. Setup is one command, and automatic capture uses Claude Code and Codex hooks. The MVP runs locally for the author alone.
No race on numbers. The benchmarks remain internal checks on quality and speed, not a leaderboard target.
Competition for chat memory. Claude's and ChatGPT's built-in memory, claude-mem, OpenMemory and Basic Memory. See section J of docs/literature/agent_memory_systems.md.
Agent Memory Atlas
How agent memories are built today, how each meets our nine requirements, and a proposal for a trained, append-only memory tape that any LLM can query in plain JSON.
What exists
Deployed agent memories (Mem0, Zep, Letta…) are prompted pipelines: an LLM call extracts text facts on every write, and off-the-shelf embeddings find them in 0.15–1 s. Nothing in them is trained.
What is trained
Trained memories live inside one model and forget by design (Titans, LM2, MemoryLLM), or plug into one reader only (xRAG, KBLaM, Cartridges). None serves an arbitrary LLM.
The gap
We found no system that trains the writer, the keys, the retriever and the offline relink pass together, keeps every event in time order, and answers any LLM through text.
Our head start
Our notebook already writes 1M tokens in under half a second, and fetching about 20 notes answers far facts in 1M-token documents (97–100 %). What's missing: a retriever that picks well on its own, time order, and an offline pass.
Literature · seven families plus ours
How agent memories are built
Every memory makes five choices: how it writes, what it stores, how it finds things, who can read them, and how it maintains itself over time. Pick a family to see its choices.
Your requirements · scored against each family
Nine requirements, nine columns
Each row is one of your requirements. Select a row to see who comes closest and why. The last column is the proposal: a design target, not a measured result.
meets it partly does not designed to meet it (not built)
Our E31 · E33 · E35 runs (30M models on Odra and Polonez)
What our own runs taught us
E35 vector messages vs E33f fetched notes
Pull beats push
Sixteen vectors composed before the question was known carried no facts (0 % on every fact task). The reader fetching 4 notes per reading round, for every answer word, answered far facts in 4k–1M-token documents at 97–100 %. The memory should answer questions, not broadcast summaries.
E35 receiver, E33 window
Easy paths starve the memory
A trainable receiver learned the question template and ignored the message. E33 kept recent text in its raw window, so near and late facts never reached the notes. Training must force answers through the memory.
notebook compression note
Notes are bigger than text
A note is 64 key plus 64 value numbers (256 bytes) per 6 tokens, about 43 bytes per token. Raw text is about 4.2. Latent notes shrink what the reader looks at (6× fewer positions, 48× smaller than a dense model's key/value cache), not the bytes on disk.
E33r probe → E33b read-back
The writer drops common words
A probe recovered a place name from notes 68 % of the time, but a property word only 11 % (guessing gives 8 %). The read-back head (each note must name its own tokens) is training now.
E33f notes-budget exam
Reads are diffuse
An answer spreads attention over dozens of notes, and the reader's first query picks the wrong ones: a single request from the first read gets 0–30 %. Picking the right notes needs its own training signal.
E33 latest task, E33e rank term
Time is the weakest skill
"Which value is the latest?" falls to 5 % at 16k tokens once the fact leaves the raw window. An untrained rank-among-matching-notes term lifts it to 41 % without hurting lookup.
Proposal · the memory tape
The proposed memory, operation by operation
Three stores, three trained parts. The tape keeps every event as exact text, in order, forever. The notes are the latent index our writer already produces. The link graph holds relations written by a scheduled sleep pass. A trained librarian runs the hops inside the memory, so a caller sends one question and gets typed answers with evidence. Pick an operation to trace its path; click any box to see what it is built from.
RequestResponse (example values)
Component
The request shape copies Jev, TypeSafe's decision model: a state plus named, typed questions; answers come back keyed by name, with probabilities and no generated text. Memory-specific types: value (the current value of something, with superseded alternatives), holds (probability a statement is true at a date), choice, when, facts (evidence to read).
Compression, bandwidth, speed
How big, how fast
Move the dials to see what a memory of a given size costs. Each figure is tagged by where it comes from: measured in our runs, vendor figure from a paper or vendor test, or estimate by arithmetic.
1M → 100M tokens of chats, documents, code
Tokens per note6 is today's reliable setting; 12 worked on 2 of 3 seeds; 48 needs wider notes
Numbers per note128 today (64 key + 64 value, 16-bit). Fewer dimensions is not quantization.
about 20 answered far facts in our 1M-token demo
The honest compression story. At today's note size the latent store is about ten times larger than the text it indexes. The memory compresses what reaches a reader: an answer reads a few kilobytes out of megabytes. Byte compression on disk only comes from more tokens per note or fewer numbers per note, the compression ladder already on our roadmap. Because the tape keeps the exact text, two memories can sync by sending text, about 4 bytes per token, and let the receiver re-run its own writer (1M tokens in half a second).
Training plan · one stage at a time, each must beat what it replaces
How to train it
Start from what already works (our page writer and E33f's notes-only questions), then add one trained part per stage. Every stage reuses our synthetic world documents, where the true answer, its source sentence and every hop of a multi-step question are known. That's what makes link and retrieval labels free.
Writer: every note says what it read
Signal
Next-word loss, plus questions that can read the document only through notes, plus read-back: each note names the tokens around its position.
A probe recovers common words (property words) from notes, not just names: well above today's 11 %.
Librarian: fetch few, fetch right
Signal
Hard top-k fetch during training (4–8 notes per hop). For each candidate note, measure how much it raises the reader's chance of the right answer; train the fetch scores to match (Atlas, REPLUG). Hard negatives: same subject but another attribute, superseded values, other documents. Questions share no words with the fact (the NoLiMa lesson).
Reuses
E33's loop becomes the librarian; E33f's checkpoint as warm start.
Pass
≥ 90 % far lookup at 1M tokens with ≤ 32 notes returned and one request per question.
Kill
No better than plain E33 given 4 notes at 16k (40 %) after 0.3B training tokens.
Time: the latest wins, history stays
Signal
Questions about the latest value, the value as of a date, whether something was ever true, and forgotten facts (answer "unknown"). Event time and status enter the fetch score; the rank-among-matching-notes term is learned.
Reuses
E33e rank term; E33d page distance; the world generator's restated facts.
Pass
Latest ≥ 60 % at 16k (today 5 %; 41 % with the untrained rank term), far lookup unchanged.
Sleep pass: links that save a hop
Signal
A relation model scores pairs (new note, candidate older note) into typed edges. It is trained so that following its edges lets a two-step question succeed in one fetch round. Labels come from the world generator's hop chains, plus "fetched together in a correct answer" edges from usage logs.
Reuses
E32 two-level notebook idea for summary notes; E33o per-hop read targets.
Pass
Two-step compose with one fetch round at least matches today's four-round score (60 % at 4k, 76 % at 16k).
Typed answers: Jev-style heads
Signal
Small heads on the librarian's state answer value, holds, choice and when, trained on the same synthetic questions, then calibrated so 70 % confidence is right 70 % of the time.
Reuses
Exact-answer scoring from the text checks.
Pass
Value accuracy equals lookup accuracy; calibration error ≤ 5 points.
Serve any LLM
Signal
Two frozen open LLMs from different families read our returned spans and facts. Retrieval is re-tuned on one frozen LLM's answer likelihood; AAR shows preferences learned on a small reader transfer to large ones. A per-reader projector (the xRAG recipe) only if latent input is wanted.
Pass
A foreign LLM reading ≤ 2k tokens from us matches the same LLM reading the full document at 128k, on our world documents and LongMemEval.
Shortcut guards in every stage
Questions can reach their document only through the memory: no raw window across the question boundary.
Controls in every exam: the memory removed, and another document's memory. If the score holds, the memory was decoration.
Answer loss only on words that need the memory, so learning the question template earns nothing.
One stage with the reader frozen, the setting that opened the channel in E35's overfit check.
Exams: our text checks (quote, lookup, keyed, latest, compose, count, deduce) from 4k to 1M tokens; LongMemEval-M (about 1.5M tokens); BEAM at 10M tokens; MemoryAgentBench's forgetting tasks; LoCoMo temporal questions. Every query also logs time taken, bytes in and bytes out.
Answers to your questions
Your four questions
1 · Any tokenizer, or embedded text?
The memory owns its tokenizer. Callers send text inside JSON, so their tokenizer never crosses the interface and any LLM works. Don't feed one LLM's embeddings into the writer: that ties the memory to that model, and every cross-model latent result we found needs an adapter trained for each model pair. If embedded queries are wanted, accept vectors from one public embedding model through a small trained adapter. Train on text: world documents, synthetic multi-session chats, code.
2 · Compose it from parts?
Yes, as the first step and the baseline. Assemble an off-the-shelf version: 64-token chunks, an open embedding model, a full-precision graph index, timestamps with a recency term, links from shared names, and a small LLM answering typed questions. Then swap in our trained parts one at a time (writer, librarian, sleep pass); each must beat the part it replaces. A frozen open LLM serves as the teacher that scores which notes help.
3 · How to build the retriever?
Two stages. First a fast shortlist: nearest-neighbour search over trained note keys, plus the notes linked to the best ones, about 256 candidates in well under a millisecond. Then the librarian reads the shortlist with attention, updates its question and hops again (1–4 times). Train it with hard top-k fetching, the reader-likelihood signal and hard negatives. When the writer changes, the sleep pass re-makes the stored keys: stale keys cost REALM 10 points.
4 · Has someone built this?
Pieces exist; the whole does not. M+ has a latent store with a co-trained retriever and age tags, but it lives inside Llama and finds about 30 % of what it needs. MemoRAG splits a memory model from the answering model. Zep keeps two clocks and never deletes. HippoRAG links facts in a graph for multi-hop. Letta runs offline "sleep-time" passes. Memory Decoder plugs into any LLM that shares its tokenizer. All of these are prompted, tied to one model, or untrained in the parts we want trained.
Propositions · ranked
What we propose to build
Split the memory into tape, notes and links
The tape is the record: exact text and every status change, in order, never rewritten. Notes and links are derived views that can be rebuilt. This gives "never forget", lossless quotes for any LLM, and room to retrain the writer without losing data.
Move the hops into the memory
The librarian (E33's loop as a service) runs multi-step fetches next to the notes. A caller sends one question of well under a kilobyte, not 22–29 KB of per-word queries as in E33f, and that one change also makes the memory usable by LLMs that cannot speak latents.
Typed questions in, calibrated answers out
A Jev-style protocol: named, typed questions (value, holds, choice, when, facts) with a state and a time ("as of"). Answers carry probabilities, superseded alternatives and evidence ids. Simple lookups need no decoder at all.
Two clocks, and change as an event
Every note carries event time and ingest time. Update appends a new event plus a "supersedes" edge; forget appends a tombstone. Reads see the latest by default and history on request. A separate admin-only purge rewrites tape segments for legal erasure.
A trained sleep pass
On a schedule, a relation model links new notes to older ones (same subject, next hop, supersedes), writes summary notes over clusters and time windows, re-keys notes after writer updates, and repairs the search graph around tombstones. It adds records and never deletes.
Measure compression three ways
Bytes stored per token, bytes sent per answer, and positions a reader attends to. Shrink notes by fewer dimensions and more tokens per note, never by quantization. Two memories sync by sending tape text and re-encoding it locally.
Train in stages with shortcut guards
Writer → librarian → time → sleep pass → typed heads → serving foreign LLMs, as above. Each stage has a pass mark and a kill condition, and every exam includes the no-memory and other-document controls.
Build the off-the-shelf baseline first
About a week of engineering. Without it we can't claim the trained parts are worth their training cost. It also gives the protocol and the exams something to run on from day one.
First bet · proposed E36, "the librarian"
Train a retriever that runs the hops inside the memory, and let a foreign LLM ask in plain text.
Claim. A small trained librarian over our existing notes, with hard top-k fetching and reader-likelihood training, returns ≤ 32 notes that answer far facts in 1M-token documents as well as E33f's per-word fetching (97–100 %), from a single text request, in under 100 ms on CPU.
Test. Warm start from the E33f model on Polonez; our world documents from 4k to 1M tokens; the controls; a frozen open LLM reading the returned spans; time and bytes logged per query.
Kill. At 16k tokens with 4 notes fetched, no better than plain E33 (40 % far lookup) after 0.3B training tokens.
Cost. Two to three Polonez bursts for training, plus a laptop pipeline check first. The spec is not written yet.
Sources · the full annotated list is in the repo's agent memory literature review