# KB Arena

Compare retrieval architectures on the same documentation and questions, then choose from recorded quality, latency, cost, and limits.

Try the demo View benchmarks GitHub

## How it works

1

### Same question

Send each question, from direct lookups to multi-topic chains, to the selected strategies.

2

### 4-pass evaluation

Structural checks, entity coverage, source attribution against your docs, then LLM-as-judge scoring.

3

### Ranked report

Accuracy by tier, latency percentiles, reliability rates, and cross-strategy composite ranking across your documentation.

## Strategy catalog

### Naive Vector

Chunks documents, embeds each chunk, and retrieves top-k by cosine similarity as a dense baseline.

Runtime status unavailable

### Contextual Vector

Prepends parent topic context to each chunk before embedding so you can measure whether document context changes retrieval.

Runtime status unavailable

### QnA Pairs

Generates question-answer pairs at index time and retrieves against those pairs instead of only source chunks.

Runtime status unavailable

### Knowledge Graph

Extracts entities and relationships into Neo4j, then queries the graph through intent-matched Cypher templates.

Runtime status unavailable

### Hybrid

Routes by intent between vector and graph paths, then uses reciprocal rank fusion when both paths contribute.

Runtime status unavailable

### RAPTOR

Builds a recursive tree of chunk clusters and summaries, then queries leaf and summary levels together.

Runtime status unavailable

### PageIndex

Builds a hierarchical tree from document structure and uses model-guided traversal without an embedding index.

Runtime status unavailable

### BM25

Uses BM25 keyword matching as a keyless lexical baseline with no embeddings or graph service.

Runtime status unavailable

### Rerank Vector

Naive Vector retrieves a wide candidate pool, then a cross-encoder reranker (BGE, Cohere, or Voyage) rescores and keeps the top-k for a measured latency-quality tradeoff.

Runtime status unavailable

### QISS (quantum)

Rescores dense candidates with a pure-NumPy state-fidelity calculation and offers an experimental multi-query mode.

Runtime status unavailable | Experimental

### SQR (optional quantum)

Experimental Qiskit Aer SWAP-test reranker. It is excluded from the default benchmark and needs the optional quantum dependency group.

Runtime status unavailable | Experimental

## 5 difficulty tiers, auto-generated or hand-crafted

### AWS Compute

Not labeled

Tier 1: Factoid Tier 2: Procedural Tier 3: Comparative Tier 4: Relational Tier 5: Multi-hop

## Built with

Python 3.11+Pydantic v2 FastAPI Neo4j 5 ChromaDB Anthropic Claude OpenAI Embeddings Next.js 16 Tailwind CSS Recharts