Loaded 27 instances from benchmarks/opendb/benchmark/codemem_dataset.json



=== LongMemEval-S Results (k=5) ===

Questions evaluated: 27

R@5:  0.5469

NDCG@5: 0.8050

Store:  2761.0 docs/s (1.1s)

Recall: 0.2 q/s (110.2s)



--- By Question Type ---

  code-architecture               n=  6  R@5=0.5556  NDCG@5=0.6667

  code-convention                 n=  4  R@5=0.7417  NDCG@5=0.9093

  code-location                   n=  4  R@5=0.4167  NDCG@5=0.5891

  knowledge-update                n=  4  R@5=0.3958  NDCG@5=0.9698

  api-signature                   n=  3  R@5=0.5000  NDCG@5=0.6609

  bug-fix                         n=  3  R@5=0.8222  NDCG@5=0.9597

  multi-session                   n=  3  R@5=0.4167  NDCG@5=1.0000



Results saved to benchmarks/opendb/benchmark/codemem_result_v2.json


