Loaded 27 instances from benchmarks/opendb/benchmark/codemem_dataset.json



=== LongMemEval-S Results (k=5) ===

Questions evaluated: 27

R@5:  0.5241

NDCG@5: 0.7856

Store:  2750.0 docs/s (1.1s)

Recall: 0.2 q/s (112.6s)



--- By Question Type ---

  code-architecture               n=  6  R@5=0.5278  NDCG@5=0.6465

  code-convention                 n=  4  R@5=0.6500  NDCG@5=0.8440

  code-location                   n=  4  R@5=0.4167  NDCG@5=0.5891

  knowledge-update                n=  4  R@5=0.3750  NDCG@5=0.9255

  api-signature                   n=  3  R@5=0.5000  NDCG@5=0.6732

  bug-fix                         n=  3  R@5=0.8222  NDCG@5=0.9597

  multi-session                   n=  3  R@5=0.4167  NDCG@5=1.0000



Results saved to benchmarks/opendb/benchmark/codemem_result_v3.json


