Local-first AI routing.
Zero cloud spend for 85% of queries.
// Route. Cache. Execute. At the speed of light.
What We Build
A four-tier routing pipeline that classifies every query and dispatches it to the cheapest capable model. Groundtruth lookups resolve deterministic queries in microseconds. Cloud only when it matters.
DeBERTa-v3 routing classifier with 44MB footprint runs entirely on CPU. Groundtruth regex resolves deterministic queries before any model wakes up.
85% of queries route to the free local tier. ChromaDB semantic cache avoids redundant calls. Cloud API cost is microscopic — fractions of a cent per session.
82.4% routing accuracy from DeBERTa fine-tuned on 5,161 labeled examples. 100% pass@1 on HumanEval. RouterBench results published openly.
Live Demo
This widget routes your query through the same pipeline powering production. Watch which tier handles it and how fast.
Tech Stack
Every component was chosen for low latency, zero unnecessary dependencies, and the ability to run entirely offline on commodity hardware.