98.94% Any@5.
99.57% Any@10.

ContextFit with optional OpenAI fusion reaches 96.6% Any@5 and 98.7% Any@10 evidence retrieval on LongMemEval-S; auditable evidence certificates and route-gated turn-aware chunk-vector fusion lift the same local retrieval harness to 98.94% Any@5 and 99.57% Any@10, with no vector database required.

98.94%Any evidence @5
99.57%Any evidence @10
87.45%All evidence @5
0.9086MRR
This is a retrieval/evidence-ranking result, not an official end-to-end LongMemEval QA score. The base fusion run uses OpenAI text-embedding-3-small embeddings as an optional cached signal; the latest run selectively embeds turn-aware conversation chunks for route types where that signal helped in validation. The evidence-certificate layer remains an auditable post-retrieval reranker with generic reason codes. No vector database is required.
Companion end-to-end QA progress: 87.2% overall accuracy and 87.6% task-averaged accuracy with ContextFit selective-fusion retrieval, a GPT-5-mini answerer/extractor, and GPT-4o judging.

Result

MetricScore
Scored examples470
Abstention examples skipped30
Any evidence @185.32%
Any evidence @395.11%
Any evidence @598.94%
Any evidence @1099.57%
All evidence @587.45%
All evidence @1091.91%
MRR0.9086

Configuration

LongMemEval-S cleaned, 500 total rows, 470 scored after abstention exclusion. ContextFit used hybrid retrieval, parent/child conversation chunks, session ranking, coverage reranking, structured temporal filters, optional OpenAI fusion, route-gated chunk-vector fusion, and evidence-certificate reranking with typed rescue. Answer markers were disabled. The selective route used full-session OpenAI vectors for 340 scored rows and conversation-chunk max vectors for 130 scored rows.

Paired Movement

Versus the prior full-session fusion + certificates + typed rescue artifact, the selective chunk-vector run produced +3 / 0 paired Any@5 movement. Complete-evidence All@5 moved +6 / -2, so the All@5 lift is useful but should be described with that tradeoff.

Reproduction Command

.venv/bin/python benchmarks/longmemeval_contextfit.py \
  benchmarks/data/longmemeval_s_cleaned.json \
  --limit 0 \
  --method hybrid \
  --top-k-chunks 10 \
  --retrieval-k 100 \
  --chunk-size 2048 \
  --overlap 128 \
  --rank-by-session \
  --conversation-chunks \
  --conversation-parent \
  --coverage-rerank \
  --structured-temporal-filters \
  --openai-fusion \
  --openai-chunk-fusion selective \
  --fusion-certificate-promotion \
  --fusion-typed-rescue \
  --fusion-final-candidate-k 80 \
  --out benchmarks/longmemeval_fusion_selective_chunk_promotion_v5_typed_rescue_20260524.json

Artifact

Raw JSON artifact: benchmarks/longmemeval_fusion_selective_chunk_promotion_v5_typed_rescue_20260524.json

SHA-256: ababf7387cb18c9310e82c35a57594aeef3cd40a2b60d9a1f336f69b65d3dcd2

GitHub report: benchmarks/longmemeval_contextfit_report.md

Recommended Wording

ContextFit with optional route-gated OpenAI chunk-vector fusion plus auditable evidence-certificate reranking reaches 98.94% Any@5 and 99.57% Any@10 evidence retrieval on LongMemEval-S, with no vector database required and zero paired Any@5 losses versus the full-session fusion certificate baseline.