F26 adds options to kin ask and conversation digests; every one is off by default, so kin ask answers LoCoMo and LongMemEval-S as F25 did (rerun on F26: 95.13 and 97.80) and BEAM-1M as F25 did (80.69 on all 700 questions). On BEAM by history size, default kin ask scores 79.11 at 100K, 74.76 at 500K and 80.69 at 1M. Every kin ask score in this summary and the charts below answers from memory alone: no dataset annotations in the prompt, and no reading of the memory as of a turn the dataset cites.
kin ask F25gpt-6-luna, full historyOriginal kin ask
Accuracy against input tokens per question (all model calls together), linear axis per benchmark; same answer model (gpt-6-luna) and judges. *The 160 BEAM-1M questions all three answered (gpt-6-luna read the latest ~800k tokens); F25 on all 700: 80.69. Measured on F25; F26 leaves the default settings unchanged.
Against published systems
kin ask F26past.devExabase M-1HindsightMem0Supermemory
Each system's own published score; n/r = not reported. Answer models differ (Exabase M-1: Gemini 3 Flash; Supermemory: Gemini 3 Pro; past.dev: gpt-5.6-luna; kin ask: gpt-6-luna). †Scored against community-corrected answer keys (kin ask: the original keys). *Not comparable: that system's published BEAM score reads the test set's grading annotations in its answer prompt (the topics to order, the rubric points the grader checks a summary against, the preference or instruction under test, why a question may be unanswerable). kin ask's BEAM-1M score uses no annotations (nugget rubric on Gemini 3 Flash).
kin ask (default settings)past.dev*Exabase M-1*HindsightMem0
BEAM by history size. kin ask: default settings, no annotations, nugget rubric on Gemini 3 Flash (not run at 10M). *Not comparable: past.dev's and Exabase's published BEAM scores read the test set's grading annotations in the answer prompt (e.g. the topics to order).
Tokens
kin ask F25gpt-6-luna, full history
Estimated input per question, all model calls together; linear, each benchmark on its own axis. Right: token reduction. kin ask F25 never exceeds 20k. *BEAM-1M: the 160-question sample.Relative accuracy gain over gpt-6-luna with the full history. *BEAM: the 160-question sample.
By category
kin ask F25gpt-6-luna, full historyOriginal kin ask
LoCoMo, 1,540 questions.
kin ask F25gpt-6-luna, full historyOriginal kin ask
LongMemEval-S, 500 questions (abstention questions counted in their type).
kin ask F25gpt-6-luna, full history
BEAM-1M abilities on the 160-question sample both systems answered.
Annotation-assisted runs (not memory results)
past.dev and Exabase publish BEAM scores whose answer prompt is built from the dataset's own per-question annotations: the topics to order, the rubric points the grader checks a summary against, the preference or instruction under test, and why a question may be unanswerable. Those annotations are the test set's grading material, so a score that reads them does not measure memory and is not comparable to the kin ask results above.
During this work kin ask was also run that way, with caller-hint code that is not part of this release: each question's annotations and Exabase's per-category guidance were passed as the answer instructions, the topics to order as extra searches, and knowledge-update questions were read as of the update turn the dataset cites; answers came from gpt-5.6-luna at reasoning effort none and were scored with past.dev's judge. The figures are recorded for reference only.
BEAM
kin ask, default (no annotations)
kin ask with annotations
seeds
past.dev with annotations, published
past.dev with annotations, same judge run
100K
79.11
91.66
91.89 / 91.57 / 91.51
92.08
91.39
500K
74.76
89.57
89.67 / 90.09 / 88.94
89.63
88.87
1M
80.69
90.61
90.23 / 90.60 / 90.99
90.65
90.17
10M
n/r
84.15
84.04 / 84.61 / 83.80
85.03
84.27
The last column scores past.dev's published answers with the same judge run (past.dev's scoring code, rubric items and judge model); there they score 0.5–0.8 below past.dev's published figures.
Notes
Judges follow each benchmark's target protocol: LoCoMo, Mem0's accuracy prompt on gpt-4o-mini; LongMemEval-S, the unified judge on Gemini 3 Flash; BEAM, the nugget rubric on Gemini 3 Flash. The annotation-assisted BEAM figures in the previous section use past.dev's judge: Exabase's rubric prompts on gpt-5.6-luna, reasoning off, items scored 0 / 0.5 / 1, event ordering by Kendall tau-b.
Turn-level claims and the new evidence options are off by default. With claims in place of facts, default kin ask scored lower on LoCoMo (93.44), LongMemEval-S (96.40) and BEAM-1M (79.56); the BEAM 1M and 10M claim stores were built with the research version of the claims prompt, which conversations.claims generalises to named speakers.
Token and category figures are F25's; F26 leaves the default settings unchanged.
Answer-key errors: 29 of F25's 73 seed-0 LoCoMo losses (about 34 of 83 averaged over three seeds) are on questions the public LoCoMo audit marks wrong.