Scored the way the Mem0 paper scored Mem0, Zep and the others, cogito-mem answers 74.7 % of LoCoMo's 1,540 questions from a 1,000-token budget of stored turns, and on the same questions that is statistically level with handing the model the whole conversation (76.5). Published under the same protocol: Mem0 66.9, Mem0g 68.4, Zep 66.0 as Mem0 ran it and 75.1 as Zep ran it. It stores without an LLM and answers a query in about 10 ms on this laptop.
Every bar below was scored the way the Mem0 paper scored its table: the memory hands context to gpt-4o-mini, which answers; gpt-4o-mini then marks the answer CORRECT or WRONG against LoCoMo's gold answer, over the 1,540 questions of categories 1–4. We ran the full-context baseline the same way, so our harness can be checked against the paper's 72.9.
Self-reported means the vendor ran its own system. Rows marked 3rd party come from the Mem0 paper, where Mem0 configured Zep, LangMem, ChatGPT memory and A-Mem; Zep disputed its row. Our bars are single runs; with 1,540 questions a run's 95 % interval is about ±2.2 points (±3 for the 762-question full-context run). Text only drops the photo captions, as Mem0's harness did; it changes nothing (−0.1 points, 95 % interval −1.6 to +1.4). Hover a bar for its context size.
Absolute scores move with the harness. With the same model, prompts and judge, our full-context run scored 76.5 on the first five conversations, where the paper reported 72.9 over all ten. Dividing each system by the full-context score measured alongside it removes that offset.
Context is counted as the answering model counts it: billed prompt tokens minus the fixed instructions, so dates and speaker names are included. cogito-mem gains 9.4 points from the 300 to the 1,000-token budget (paired 95 % interval 7.1 to 11.6) and 2.3 more at 2,500 (0.5 to 4.0). Every system scored with the same prompts sits below the cogito-mem line at its own context size. MemOS (*), with its own answer prompt and an unnamed judge, reports 75.8 at about 1,600 tokens.
Single-hop questions, one fact said once, are where returning the original turns pays off: 84.5 at 1,000 tokens against 72.9 for Mem0. Temporal questions do well because every returned turn carries its date; the whole conversation does worst there (59.6), as the model loses track of which session said what. Multi-hop and open-domain questions need several turns or an inference across the conversation, and that is where a fixed budget runs out first. When every turn an answer needs was returned, gpt-4o-mini answered 85.6 % correctly at 1,000 tokens; when one was missing, 45.7 %.
Categories use the dataset's own ids: 1 multi-hop (282), 2 temporal (321), 3 open-domain (96), 4 single-hop (841). The Mem0 paper attached these names to different ids; its row here is re-mapped by question count, so it may differ from the paper's printed columns.
A query over one LoCoMo conversation takes about 10 ms. Over 10M tokens the 95th percentile is 26 ms on the CPU alone, a quarter of the 100 ms target in the requirements. Hosted memories report 0.1 to 3 s, largely network and their own model calls, so this compares architectures, not hardware. Appending returns in 1 to 11 ms; embedding follows in the background at about 21k tokens a second, so 10M tokens of history is indexed in about 8 minutes.
Since late 2025 most vendors report LoCoMo with stronger answering models (GPT-4.1-mini, GPT-5, Gemini 3), longer step-by-step answer prompts and more lenient judges. Those numbers measure the model and the prompt as much as the memory, and they are not comparable with the table above or with each other.
.mcp.json failed to start here: it runs ${CLAUDE_PLUGIN_ROOT}/scripts/cogito-mem-plugin, and that variable is set only when the repo is installed as a plugin, not when it is opened as a project.memory_query in 10 to 22 ms, memory_append in under 1 ms.trial memory: 480 messages and 194k tokens appended in half a second, embedded in the background within seconds.docs/requirements.md), which capture skips on purpose.cogito-mem setup could not install llama.cpp on any platform, because llama.cpp's newest GitHub release (v0.7.0) carries no binaries; new installs were silently keyword-only.cogito-mem install claude-code: a user-scope MCP server plus the Stop and SessionEnd capture hooks; then cogito-mem capture claude-code --all for the history.UserPromptSubmit hook queries the memory with each prompt (about 15 ms) and adds the hits above a score threshold as context. With many MCP servers, Claude Code defers their tools behind tool search, so a recall tool that waits to be called is rarely used unprompted.SessionStart hook adds the last few exchanges of the previous session in the same project.cwd and git branch (the transcript has both; captured items have no title today) and prefer the current project when ranking.memory_append, as the server's instructions already ask.locomo10.json from snap-research/locomo (sha256 79fa87e9…): 10 conversations, 1,540 questions in categories 1–4; the 446 adversarial questions are left out, as everyone does.top_k 100, max_tokens 300, 1,000 or 2,500.openai/gpt-4o-mini through OpenRouter, temperature 0, with Mem0's prompts word for word: the memory prompt Mem0 used for Zep, the paper's full-context prompt, and its judge prompt in JSON mode. Returned turns are shown in date order as "date: speaker: text".cogito-mem bench speed, 1,000 queries one at a time, top_k 10, 300 tokens. Every query is embedded by the real model; the stored 10M tokens are synthetic text with random vectors (embedding them for real is timed separately, on 1M tokens).cogito-mem eval locomo-qa --data locomo10.json --env-file .env --context memory --budget 1000 --out results/locomo-qa/paper-mem1000.json cogito-mem eval locomo-qa --data locomo10.json --env-file .env --context full --out results/locomo-qa/paper-full.json cogito-mem eval locomo-qa compare results/locomo-qa/paper-full.json results/locomo-qa/paper-mem1000.json COGITO_MEM_GPU_LAYERS=0 cogito-mem bench speed --tokens 10M --random-vectors --out results/m5max/speed-10M-cpu.json