Generated: 1771918404s-since-epoch
| Metric | No Ctx | Know+CLI | CLI Search | MCP+Tools | MCP Only |
|---|---|---|---|---|---|
| Total Recall | |||||
| Avg Response Time | 29.2s | 60.0s | 60.0s | 60.0s | 60.0s |
| Success Rate | 10/10 | 0/10 | 0/10 | 0/10 | 0/10 |
| Timeouts | 0 | 10 | 10 | 10 | 10 |
| Query | No Ctx | Know+CLI | CLI Search | MCP+Tools | MCP Only |
|---|---|---|---|---|---|
| q-001What is the Zephyr Query Protocol and how does its three-phase handshake work? | 0% | TIMEOUT | TIMEOUT | TIMEOUT | TIMEOUT |
| q-002How does ConpackDB implement ring-buffer sharding with BLAKE3 partition keys? | 100% | TIMEOUT | TIMEOUT | TIMEOUT | TIMEOUT |
| q-003Explain the Meridian consensus protocol used by Nexara and its temporal sharding approach | 0% | TIMEOUT | TIMEOUT | TIMEOUT | TIMEOUT |
| q-004How do you configure the Vortex ingestion pipeline stages and dead-letter handling? | 100% | TIMEOUT | TIMEOUT | TIMEOUT | TIMEOUT |
| q-005What traffic shaping policies does the Auralis service mesh support? | 100% | TIMEOUT | TIMEOUT | TIMEOUT | TIMEOUT |
| q-006How do you train custom embeddings with the Helix Embedder and export to ONNX? | 0% | TIMEOUT | TIMEOUT | TIMEOUT | TIMEOUT |
| q-007Describe the Prismoid cache coherence protocol and its gossip-based invalidation | 0% | TIMEOUT | TIMEOUT | TIMEOUT | TIMEOUT |
| q-008What resources does the conpack-cluster Terraform module provision and what are the auto-scaling triggers? | 0% | TIMEOUT | TIMEOUT | TIMEOUT | TIMEOUT |
| q-009How does the Spectral Query Analyzer optimize execution plans for multi-upstream searches? | 0% | TIMEOUT | TIMEOUT | TIMEOUT | TIMEOUT |
| q-010Explain the Obsidian Vault Sync protocol modes and delta sync mechanism for knowledge files | 100% | TIMEOUT | TIMEOUT | TIMEOUT | TIMEOUT |
| Vertical | Doc | Match | Confidence | Time | Matched Terms |
|---|---|---|---|---|---|
| No Ctx | eval-001 | NO | 0.205 | 11.3s | |
| Know+CLI | eval-001 | NO | 0.000 | 60.0s | |
| CLI Search | eval-001 | NO | 0.000 | 60.0s | |
| MCP+Tools | eval-001 | NO | 0.000 | 60.0s | |
| MCP Only | eval-001 | NO | 0.000 | 60.0s |
No response captured
No response captured
No response captured
No response captured
| Vertical | Doc | Match | Confidence | Time | Matched Terms |
|---|---|---|---|---|---|
| No Ctx | eval-002 | YES | 0.432 | 15.4s | |
| Know+CLI | eval-002 | NO | 0.000 | 60.0s | |
| CLI Search | eval-002 | NO | 0.000 | 60.0s | |
| MCP+Tools | eval-002 | NO | 0.000 | 60.0s | |
| MCP Only | eval-002 | NO | 0.000 | 60.0s |
No response captured
No response captured
No response captured
No response captured
| Vertical | Doc | Match | Confidence | Time | Matched Terms |
|---|---|---|---|---|---|
| No Ctx | eval-003 | NO | 0.250 | 22.5s | |
| Know+CLI | eval-003 | NO | 0.000 | 60.0s | |
| CLI Search | eval-003 | NO | 0.000 | 60.0s | |
| MCP+Tools | eval-003 | NO | 0.000 | 60.0s | |
| MCP Only | eval-003 | NO | 0.000 | 60.0s |
No response captured
No response captured
No response captured
No response captured
| Vertical | Doc | Match | Confidence | Time | Matched Terms |
|---|---|---|---|---|---|
| No Ctx | eval-004 | YES | 0.364 | 49.9s | |
| Know+CLI | eval-004 | NO | 0.000 | 60.0s | |
| CLI Search | eval-004 | NO | 0.000 | 60.0s | |
| MCP+Tools | eval-004 | NO | 0.000 | 60.0s | |
| MCP Only | eval-004 | NO | 0.000 | 60.0s |
No response captured
No response captured
No response captured
No response captured
| Vertical | Doc | Match | Confidence | Time | Matched Terms |
|---|---|---|---|---|---|
| No Ctx | eval-005 | YES | 0.364 | 19.5s | |
| Know+CLI | eval-005 | NO | 0.000 | 60.0s | |
| CLI Search | eval-005 | NO | 0.000 | 60.0s | |
| MCP+Tools | eval-005 | NO | 0.000 | 60.0s | |
| MCP Only | eval-005 | NO | 0.000 | 60.0s |
No response captured
No response captured
No response captured
No response captured
| Vertical | Doc | Match | Confidence | Time | Matched Terms |
|---|---|---|---|---|---|
| No Ctx | eval-006 | NO | 0.273 | 34.9s | |
| Know+CLI | eval-006 | NO | 0.000 | 60.0s | |
| CLI Search | eval-006 | NO | 0.000 | 60.0s | |
| MCP+Tools | eval-006 | NO | 0.000 | 60.0s | |
| MCP Only | eval-006 | NO | 0.000 | 60.0s |
No response captured
No response captured
No response captured
No response captured
| Vertical | Doc | Match | Confidence | Time | Matched Terms |
|---|---|---|---|---|---|
| No Ctx | eval-007 | NO | 0.250 | 11.0s | |
| Know+CLI | eval-007 | NO | 0.000 | 60.0s | |
| CLI Search | eval-007 | NO | 0.000 | 60.0s | |
| MCP+Tools | eval-007 | NO | 0.000 | 60.0s | |
| MCP Only | eval-007 | NO | 0.000 | 60.0s |
No response captured
No response captured
No response captured
No response captured
| Vertical | Doc | Match | Confidence | Time | Matched Terms |
|---|---|---|---|---|---|
| No Ctx | eval-008 | NO | 0.159 | 53.2s | |
| Know+CLI | eval-008 | NO | 0.000 | 60.0s | |
| CLI Search | eval-008 | NO | 0.000 | 60.0s | |
| MCP+Tools | eval-008 | NO | 0.000 | 60.0s | |
| MCP Only | eval-008 | NO | 0.000 | 60.0s |
No response captured
No response captured
No response captured
No response captured
| Vertical | Doc | Match | Confidence | Time | Matched Terms |
|---|---|---|---|---|---|
| No Ctx | eval-009 | NO | 0.273 | 15.0s | |
| Know+CLI | eval-009 | NO | 0.000 | 60.0s | |
| CLI Search | eval-009 | NO | 0.000 | 60.0s | |
| MCP+Tools | eval-009 | NO | 0.000 | 60.0s | |
| MCP Only | eval-009 | NO | 0.000 | 60.0s |
No response captured
No response captured
No response captured
No response captured
| Vertical | Doc | Match | Confidence | Time | Matched Terms |
|---|---|---|---|---|---|
| No Ctx | eval-010 | YES | 0.523 | 59.5s | |
| Know+CLI | eval-010 | NO | 0.000 | 60.0s | |
| CLI Search | eval-010 | NO | 0.000 | 60.0s | |
| MCP+Tools | eval-010 | NO | 0.000 | 60.0s | |
| MCP Only | eval-010 | NO | 0.000 | 60.0s |
No response captured
No response captured
No response captured
No response captured