kin ask F25gpt-6-luna, full historyOriginal kin ask
Accuracy against input tokens per question (all model calls together), linear axis per benchmark; same answer model (gpt-6-luna) and judges. *The 160 BEAM-1M questions all three answered (gpt-6-luna read the latest ~800k tokens); F25 on all 700: 80.69.
Against published systems
kin ask F25past.devExabase M-1HindsightMem0Supermemory
Each system's own published score; n/r = not reported. Answer models differ (Exabase M-1: Gemini 3 Flash; Supermemory: Gemini 3 Pro; kin ask: gpt-6-luna). †Scored against community-corrected answer keys (kin ask: the original keys). ‡Published BEAM answer prompts include the dataset's annotations, such as the topics to order; kin ask uses none.
kin ask F25past.devExabase M-1HindsightMem0
BEAM by history size. kin ask was not run at 10M. past.dev's and Exabase's published BEAM answer prompts include the dataset's annotations (e.g. the topics to order); kin ask uses none.
Tokens
kin ask F25gpt-6-luna, full history
Estimated input per question, all model calls together; linear, each benchmark on its own axis. Right: token reduction. kin ask F25 never exceeds 20k. *BEAM-1M: the 160-question sample.Relative accuracy gain over gpt-6-luna with the full history. *BEAM: the 160-question sample.
By category
kin ask F25gpt-6-luna, full historyOriginal kin ask
LoCoMo, 1,540 questions.
kin ask F25gpt-6-luna, full historyOriginal kin ask
LongMemEval-S, 500 questions (abstention questions counted in their type).
kin ask F25gpt-6-luna, full history
BEAM-1M abilities on the 160-question sample both systems answered.
Notes
Judges follow each benchmark's target protocol: LoCoMo, Mem0's accuracy prompt on gpt-4o-mini; LongMemEval-S, the unified judge on Gemini 3 Flash; BEAM, the nugget rubric on Gemini 3 Flash.
Answer-key errors: 29 of F25's 73 seed-0 LoCoMo losses (about 34 of 83 averaged over three seeds) are on questions the public LoCoMo audit marks wrong.