Router Benchmark
Live test — muLLM routing accuracy, latency & cost vs cloud
🌲 3D Viz
Pass Rate
—
Avg Latency
—
Total Cost
—
% Local
—
Problems Run
0
muLLM Benches
100%
HumanEval
164/164 problems · local · $0
Q: has_close_elements([1.0, 2.0, 3.9, 4.0], 0.3)?
→ True  ·  def has_close_elements(nums, t): ...
98.3%
MultiPL-E Code
5 languages · 120 problems · $0
Q: JavaScript — is_palindrome(s)
const isPalin = s => s === [...s].reverse().join('')
60%
GSM8K Math
math reasoning · local · $0
Q: 15% tip on $48.60 dinner for 4, split evenly?
→ $13.97 per person  ·  (48.60×1.15)/4
…
Routing Decision
loading…
Q: "sort a list of dicts by key" → classify
→ code  ·  local  ·  61µs  ·  $0.00
99.8%
Token Cost Avoided
12,393 queries · $14.99 total · $7,121 saved vs Opus
Q: real spend vs hypothetical all-Opus cost?
$14.99 spent → $7,121 saved  ·  RouteLLM (ICLR'25) reports 70% industry avg
82.4%
RouteEval
DeBERTa vs human oracle · 200 queries · $0
Q: Does our classifier pick the right tier?
→ 82.4% matches human  ·  3/200 the 9B overruled human (correctly)
muLLM Benches (mµLO)
mµLO: —
Running… 0%
muLLM saved vs Claude Opus 4
$0.00
Problem Status Tier / Model Latency Cost
Select a tab and press Run Benchmark to start.

HumanEval — Canonical Code Benchmark
164 problems · assert-based tests · subprocess sandbox · official pass@1 metric
▶ HumanEval Sampler
20 problems · Local model only · Free · ~2 min
$0 20 problems local only ~2 min
▮▮▮ Full Suite
164 problems · 3-way comparison: Local / muLLM-Routed / Cloud-Haiku
~$3–8 164 problems 3 streams ~15 min
Re-run stream only:
⚡ Groundtruth Speed Bench
20 lookup questions · groundtruth vs local model · real latency · $0 cost
$0 20 questions in-process vs model ~10 sec
⏱ Route Latency Profiler
12 routers · routing decision in µs · accuracy · efficiency score · $0
$0 12 routers µs precision ~5 sec
⚡ RouterBench
200 problems · 5 routers compared · routing precision/recall · $0 cost
$0 200 problems 5 routers ~3 sec
Compare adds mullm+BEST-Route vs naive — slot for future strategies
Running… 0%
Press Run HumanEval Sampler to benchmark 20 HumanEval problems against the local model.

Sampler Suite — HumanEval 20 + MBPP + Routing + Latency + Cache
Local only · $0 · Real code execution · subprocess sandbox · 10s timeout per problem
Running… 0%
Routing Radar — Quality Dimensions
6-axis spider chart comparing muLLM vs always-cloud baseline · Coding = 100% HumanEval · muEval = human-reviewed routing accuracy · click vertices for numbers
muLLM (current)
Always-cloud baseline