With today's answering models (GPT-5 mini, Claude Haiku 5.5, gpt-oss-120b, Qwen3), cogito-mem answers 85.6 to 89.9 % of LoCoMo's 1,540 questions under Mem0's own 2026 judging rules. That is the level published memory systems report, and 3.4 to 4.7 points below reading the whole conversation, while the model reads 9 times fewer tokens. It stores without any model call, searches in milliseconds on a laptop, and keeps everything on your machine.
Each answering model read either what cogito-mem returned for the question, within a 1,000-token budget, or the whole conversation. The answer prompt is the one in Mem0's open benchmark (memory-benchmarks, 2026). GPT-5 mini then judged every answer twice: with Mem0's 2026 rules, which most current vendor numbers resemble, and with the stricter 2025 rules. The same 1,540 questions were used in every run.
Differences are paired over the same 1,540 questions; hover a row for the interval and the stricter judge. GPT-5 mini and gpt-oss-120b ran with low reasoning effort. The open models were pinned to hosts serving full-precision weights (gpt-oss-120b in bf16, Qwen3 in fp8). Prompt tokens include the same instructions in both arms. A single run's 95 % interval is about ±1.5 points at 90 %.
1,000 tokens is cogito-mem's budget for an explicit recall. With GPT-5 mini answering, 300 tokens scores 79.6 and 2,500 tokens scores 92.1 (about 4,900 prompt tokens), 2.0 points (95 % interval 0.8 to 3.2) below the whole conversation.
The closest published comparison is Mem0's own: its platform scored 91.6 % (its README says 92.5) with this same answer prompt and these judging rules, but with the larger GPT-5 answering and judging, and 200 memories per question. cogito-mem lands at 85.6 to 89.9 with smaller models and a 1,000-token budget. Other vendors use their own prompts, judges and models; their numbers sit in the same band but cannot be compared point for point.
The case for cogito-mem is not a higher score; it is what a score costs. No model call to store a message, about 3,400 prompt tokens to answer, local search in milliseconds, and nothing to host or sign up for.
A query over one LoCoMo conversation takes about 10 ms. Over 10M tokens the 95th percentile is 26 ms on the CPU alone, a quarter of the 100 ms target in the requirements. Hosted memories report 0.1 to 3 s, largely network and their own model calls, so this compares architectures, not hardware. Appending returns in 1 to 11 ms; embedding follows in the background at about 21k tokens a second, so 10M tokens of history is indexed in about 8 minutes.
The Mem0 paper scored every system with gpt-4o-mini answering through its own prompts and gpt-4o-mini judging. Most 2025 tables reuse its numbers. We ran cogito-mem and the full-context baseline the same way, with the paper's prompts word for word.
Self-reported means the vendor ran its own system. Rows marked 3rd party come from the Mem0 paper, where Mem0 configured Zep, LangMem, ChatGPT memory and A-Mem; Zep disputed its row. Our bars are single runs; with 1,540 questions a run's 95 % interval is about ±2.2 points (±3 for the 762-question full-context run). Text only drops the photo captions, as Mem0's harness did; it changes nothing (−0.1 points, 95 % interval −1.6 to +1.4). Hover a bar for its context size.
Absolute scores move with the harness. With the same model, prompts and judge, our full-context run scored 76.5 on the first five conversations, where the paper reported 72.9 over all ten. Dividing each system by the full-context score measured alongside it removes that offset.
Context is counted as the answering model counts it: billed prompt tokens minus the fixed instructions, so dates and speaker names are included. cogito-mem gains 9.4 points from the 300 to the 1,000-token budget (paired 95 % interval 7.1 to 11.6) and 2.3 more at 2,500 (0.5 to 4.0). Every system scored with the same prompts sits below the cogito-mem line at its own context size. MemOS (*), with its own answer prompt and an unnamed judge, reports 75.8 at about 1,600 tokens.
Single-hop questions, one fact said once, are where returning the original turns pays off: 84.5 at 1,000 tokens against 72.9 for Mem0. Temporal questions do well because every returned turn carries its date; the whole conversation does worst there (59.6), as the model loses track of which session said what. Multi-hop and open-domain questions need several turns or an inference across the conversation, and that is where a fixed budget runs out first. When every turn an answer needs was returned, gpt-4o-mini answered 85.6 % correctly at 1,000 tokens; when one was missing, 45.7 %.
Categories use the dataset's own ids: 1 multi-hop (282), 2 temporal (321), 3 open-domain (96), 4 single-hop (841). The Mem0 paper attached these names to different ids; its row here is re-mapped by question count, so it may differ from the paper's printed columns.
memory_query in 10 to 22 ms, memory_append in under 1 ms.trial memory: 480 messages and 194k tokens appended in half a second, embedded in the background within seconds.docs/requirements.md), which capture skips on purpose.memory_append stored a note in 4.9 ms, memory_query found it by meaning in 19.5 ms, and the next prompt arrived with a memory the recall hook had added on its own.cogito-mem setup could not install llama.cpp on any platform, because llama.cpp's newest GitHub release (v0.7.0) carries no binaries; new installs were silently keyword-only. The repo's .mcp.json, which failed to start when the repo was opened as a project, is gone: the MCP server now lives in the plugin's manifest.cogito-mem install claude-code: a user-scope MCP server, capture on Stop and SessionEnd, and cogito-mem capture claude-code --all for the history.UserPromptSubmit hook queries the memory with each prompt and adds up to 300 tokens of hits from earlier sessions that clear a score and a shared-word threshold, the current project's first; it gives up after 1 s rather than hold up the prompt. With many MCP servers, Claude Code defers their tools behind tool search, so a recall tool that waits to be called is rarely used unprompted.SessionStart hook adds the last exchanges of the most recent earlier session in the same project, read from its transcript.memory_append, as the server's instructions ask.locomo10.json from snap-research/locomo (sha256 79fa87e9…): 10 conversations, 1,540 questions in categories 1–4; the 446 adversarial questions are left out, as everyone does.top_k 100, max_tokens 300, 1,000 or 2,500.openai/gpt-4o-mini through OpenRouter, temperature 0, with Mem0's prompts word for word: the memory prompt Mem0 used for Zep, the paper's full-context prompt, and its judge prompt in JSON mode. Returned turns are shown in date order as "date: speaker: text".cogito-mem bench speed, 1,000 queries one at a time, top_k 10, 300 tokens. Every query is embedded by the real model; the stored 10M tokens are synthetic text with random vectors (embedding them for real is timed separately, on 1M tokens).cogito-mem eval locomo-qa --data locomo10.json --env-file .env --context memory --budget 1000 --out results/locomo-qa/paper-mem1000.json cogito-mem eval locomo-qa --data locomo10.json --env-file .env --protocol mem0-2026 --context memory --budget 1000 --answer-model openai/gpt-5-mini --answer-reasoning low --answer-max-tokens 8000 --judge-model openai/gpt-5-mini --judge-reasoning minimal --judges mem0-2026,mem0-2025 --out results/gpt-5-mini-memory1000.json cogito-mem eval locomo-qa --data locomo10.json --env-file .env --context full --out results/locomo-qa/paper-full.json cogito-mem eval locomo-qa compare results/locomo-qa/paper-full.json results/locomo-qa/paper-mem1000.json COGITO_MEM_GPU_LAYERS=0 cogito-mem bench speed --tokens 10M --random-vectors --out results/m5max/speed-10M-cpu.json