PlanWise Router — Architecture

A routing layer that picks the 3–6 right tools out of a 60+ tool catalog for each query, along with a step-by-step plan. One engine, three ways to plug it in.

Offline · Intake & Index

Runs once per catalog; ~2s for each added or changed tool. No training — new tools work right away.

Tool catalog (JSON)
Any format — OpenAI, Anthropic, MCP — converted by format adapters into one internal card.
→
Intake guardrail
Schema check · vague-description check · self-retrieval test: a tool must come up #1 for its own example questions, or it's rejected with a report naming the tools it clashes with.
→
Enrichment
Tool card incl. output fields + ~8 example user questions per tool (written by an LLM), each stored as its own vector.
→
Indexes & graphs
Meaning vectors · keyword index · ~10 capability groups · twin groups (same inputs and same outputs) · version families (v2 replaces v1).
↓indexes loaded in-memory · stateless runtime
Runtime · Route a query

One cheap LLM call + fast local math (<30 ms). Everything else is rule-based and explainable.

User query
"Pull last quarter's revenue and email the chart to finance."
→
Planner HAIKU · 1 CALL
Breaks the query into ordered abstract steps — {verb, what, needs, produces, repeat}. Sees only group names, never tool names → it can't invent a tool that doesn't exist.
↓  for each step
Cluster pre-filter LOCAL
Match the step against the ~10 groups; keep the best 2. ~80% of the catalog is out before scoring starts.
→
Hybrid scorer LOCAL
Every tool in the surviving groups is scored on four signals (below). Matching against the example questions is the main one; output_fit checks what the tool returns against what this step — or the next step — needs. That's how twins with the same inputs get told apart.
→
Bind & gate LOCAL
How many tools to keep is decided per step — a minimum score plus "must be close to the top score". Losers go to the backup pool, not the trash.
score(tool) = 0.40·semantic(MaxSim) + 0.25·BM25 + 0.15·param_fit + 0.20·output_fit
  • Clear winner (more than ~5% ahead) → pick it, exactly 1 tool.
  • Twins (same inputs + same outputs) → keep the higher-scored one; the other → backup pool.
  • Versions → newest wins, unless the query asks for the old one or the new one needs an input we can't fill; loser → backup pool.
  • Two different tools almost tied → send both (advisory) / pick the top one (strict).
  • Nothing scores high enough → search wider → check if the step needs no tool → one re-plan → ask the user. Logged as a missing-capability gap.
↓  union across steps
Assembly
3–6 tools, sorted in the order the steps need them, + a short plan hint: 1) get_revenue_report 2) generate_chart 3) send_email. ~90% fewer tool tokens than the full catalog.
↓filtered tools + plan
Execution · Self-healing middleware

Wraps the agent's loop. Optional — Tier 1 callers can run things however they like.

Strict mode INNOVATION
Shows the agent only the current step's tool; swaps in the next one when the step succeeds. Right order & count are guaranteed by the system, not hoped for.
Interceptors
Unknown tool name → find the real one in the full catalog and inject it live · tool error → swap in the backup twin or older version · request_tool() lets the agent ask for a missing tool.
Widening ladder
More candidates → neighbor groups → whole catalog → a clear question back to the user. Max 2 rounds; the session never crashes.
↓same engine behind every surface
Integration surfaces · pluggable into any system

The engine is a plain Python library; each surface is a thin layer on top. One repo, pip-installable, with Docker.

Tier 1 — HTTP gateway
POST /catalog · /route · /feedback. No stored state, docker run, works with any stack in one call. Answers in the caller's format (?format=openai|anthropic).
Tier 2 — Client wrapper
pip install planwise → PlanWiseWrapper(client). Adds self-healing + strict mode inside your loop. Optional extra, never required.
Tier 3 — MCP server
Installed with one config entry (uvx). Any MCP client (Claude Desktop, Cursor…) gets the routed catalog with zero code.
Eval harness (a deliverable — it produces the results table): plan-vs-ground-truth scoring · stress flags --add-distractors · --drop-tool · --swap-tool · --version-bump · baselines: all-tools-in-prompt and plain top-5 similarity.
offline, per catalog change runtime hot path, per query execution loop, optional integration & delivery