# Benchmark results

Accuracy by tier, latency, and cost for the strategies recorded in each run.

Corpus
Checked sample run. Use `kb-arena benchmark` to evaluate your corpus.

Strategy | Tier 1 Lookupi Single fact retrieval from one document. Example: 'What is the default timeout?' | Tier 2 How-Toi Step-by-step procedure within one topic. Example: 'How do I enable server-side encryption?' | Tier 3 Comparisoni Choosing between two options or configurations. Example: 'Compare hot storage vs cold archive for compliance.' | Tier 4 Integrationi Cross-topic dependencies requiring 3-4 connected components. Example: 'What permissions does service A need for B and C?' | Tier 5 Architecturei Full system design spanning 3-5+ topics. Example: 'How does a request flow from ingress through processing to storage?' | Avg % v | Latency | Cost/Q
--- | --- | --- | --- | --- | --- | --- | --- | ---
QnA Pairsi Generates question-answer pairs at index time and retrieves against those pairs instead of only source chunks. | 79% | 85% | 83% | 84% | 66% | 79.4% | 9043 ms | $0.4800
Knowledge Graphi Extracts entities and relationships into Neo4j, then queries the graph through intent-matched Cypher templates. | 72% | 69% | 61% | 77% | 79% | 71.6% | 20322 ms | $1.3700
Hybridi Routes by intent between vector and graph paths, then uses reciprocal rank fusion when both paths contribute. | 39% | 81% | 61% | 80% | 62% | 64.6% | 41549 ms | $3.0200
RAPTORi Builds a recursive tree of chunk clusters and summaries, then queries leaf and summary levels together. | 30% | 16% | 15% | 36% | 30% | 25.4% | 7240 ms | $0.6900
Naive Vectori Chunks documents, embeds each chunk, and retrieves top-k by cosine similarity as a dense baseline. | 27% | 15% | 14% | 26% | 22% | 20.8% | 6421 ms | $0.3300
Contextual Vectori Prepends parent topic context to each chunk before embedding so you can measure whether document context changes retrieval. | 25% | 11% | 9% | 26% | 11% | 16.4% | 5114 ms | $0.2900
PageIndexi Builds a hierarchical tree from document structure and uses model-guided traversal without an embedding index. | 19% | 12% | 7% | 21% | 12% | 14.2% | 10933 ms | $0.2900
BM25i Uses BM25 keyword matching as a keyless lexical baseline with no embeddings or graph service. | 24% | 11% | 9% | 16% | 10% | 14.0% | 4514 ms | $0.2600

Results from your benchmark runs. Run `kb-arena benchmark` to regenerate with different corpora or strategies.

## Accuracy by tier

QnA Pairs Knowledge Graph Hybrid RAPTOR Naive Vector Contextual Vector PageIndex BM25

### Methodology

Each question is sent to the strategies recorded in the run. Answers are evaluated through a 4-pass pipeline: structural checks (must_mention / must_not_claim), entity coverage against source documentation, source attribution, and LLM-as-judge scoring for accuracy, completeness, and faithfulness.

Composite ranking: 0.5 * accuracy + 0.3 * reliability + 0.2 * latency_score. Latency score inverts p95 so lower is better.

Tiers: 1 = lookup (single fact retrieval), 2 = how-to (procedure within one topic), 3 = comparison (option A vs B), 4 = integration (cross-topic dependencies), 5 = architecture (3+ topics, system design).