Loaded 27 instances from benchmarks/opendb/benchmark/codemem_dataset.json



=== LongMemEval-S Results (k=5) ===

Questions evaluated: 27

R@5:  0.6130

NDCG@5: 0.8665

Store:  2471.9 docs/s (1.2s)

Recall: 0.3 q/s (106.5s)



--- By Question Type ---

  code-architecture               n=  6  R@5=0.5278  NDCG@5=0.6465

  code-convention                 n=  4  R@5=0.7833  NDCG@5=0.9395

  code-location                   n=  4  R@5=0.5833  NDCG@5=0.7198

  knowledge-update                n=  4  R@5=0.4167  NDCG@5=1.0000

  api-signature                   n=  3  R@5=0.7778  NDCG@5=0.9597

  bug-fix                         n=  3  R@5=0.8889  NDCG@5=1.0000

  multi-session                   n=  3  R@5=0.4167  NDCG@5=1.0000



Results saved to benchmarks/opendb/benchmark/codemem_result_v4.json


