muLLM — MMLU Cost / Accuracy Pareto

Key insight: muLLM's smart routing (30B local + selective cloud escalation) targets ~78% MMLU accuracy at $0.04/1K queries — vs $2.00/1K for all-cloud GPT-4 at 86%. That's a 50× cost reduction for a 10pp accuracy trade-off. For latency-sensitive workloads where the 30B runs at 40–80 tok/s locally, the cloud RTT savings (200–800ms avoided for 85% of queries) make muLLM strictly faster as well.

Cache note: The semantic cache returns stale answers at ~43% MMLU accuracy — worse than random for adversarial MCQ. MMLU benchmarks should always use skip_cache: true. The cache shines on repeated factual queries in production (>97% hit rate on real traffic), not benchmark diversity.