Metadata-Version: 2.4
Name: toolbank
Version: 1.1.0
Summary: ToolBank - intelligent discovery and routing layer for Model Context Protocol servers. Discovery + Dynamic tool-level selection. Save tokens.
Project-URL: Homepage, https://github.com/KrzysztofAugiewicz/ToolBank
Project-URL: Repository, https://github.com/KrzysztofAugiewicz/ToolBank
Project-URL: Documentation, https://github.com/KrzysztofAugiewicz/ToolBank/tree/main/docs
Project-URL: Bug Tracker, https://github.com/KrzysztofAugiewicz/ToolBank/issues
Author: Krzysztof Augiewicz, Kacper Pisarczyk
Maintainer: Krzysztof Augiewicz
License-Expression: MIT
License-File: AUTHORS.md
License-File: LICENSE
Keywords: ai,discovery,dynamic,mcp,model-context-protocol,protocol,toolbank,tools
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Requires-Python: >=3.11
Provides-Extra: all
Requires-Dist: httpx>=0.27.0; extra == 'all'
Requires-Dist: mcp>=1.0.0; extra == 'all'
Requires-Dist: numpy>=1.24.0; extra == 'all'
Requires-Dist: sentence-transformers>=2.2.0; extra == 'all'
Requires-Dist: tiktoken>=0.5.0; extra == 'all'
Provides-Extra: benchmark
Requires-Dist: tiktoken>=0.5.0; extra == 'benchmark'
Provides-Extra: dev
Requires-Dist: httpx>=0.27.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: tiktoken>=0.5.0; extra == 'dev'
Provides-Extra: embeddings
Requires-Dist: numpy>=1.24.0; extra == 'embeddings'
Requires-Dist: sentence-transformers>=2.2.0; extra == 'embeddings'
Provides-Extra: http
Requires-Dist: httpx>=0.27.0; extra == 'http'
Provides-Extra: mcp
Requires-Dist: mcp>=1.0.0; extra == 'mcp'
Description-Content-Type: text/markdown

# ToolBank

**The discovery and routing layer that keeps MCP servers out of your context window until you actually need them.**

[![CI](https://github.com/KrzysztofAugiewicz/ToolBank/actions/workflows/ci.yml/badge.svg)](https://github.com/KrzysztofAugiewicz/ToolBank/actions/workflows/ci.yml)
[![PyPI version](https://img.shields.io/pypi/v/toolbank?color=green)](https://pypi.org/project/toolbank/)
[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-green.svg)](https://python.org)
[![Coverage](https://img.shields.io/badge/coverage-94%25-brightgreen.svg)](https://github.com/KrzysztofAugiewicz/ToolBank)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/LICENSE)

---

## The problem

The Model Context Protocol lets an LLM talk to any number of servers — GitHub, Slack, Postgres, your internal tools. The catch: most clients load **every** tool definition from **every** configured server at startup. Six servers can mean 70+ tool schemas and thousands of tokens spent before the conversation even starts, most of which the model never touches in a given session.

## What ToolBank does

ToolBank sits in front of your MCP servers as a thin discovery layer. Instead of loading everything up front, the LLM asks for what it needs — by server or by tool — and ToolBank resolves the request and connects on demand. You keep your existing MCP servers unmodified; ToolBank only changes how (and when) their tools reach the model's context.

It works at the protocol level, so it doesn't care what's actually behind a server — GitHub, Slack, a database, or a fully custom MCP server you built yourself over a proprietary application (CAD software, a game engine editor, an internal build system). If it speaks MCP, ToolBank can discover it and route to it. The bigger and more varied your server catalog gets, the more it pays off — see [At large scale](#at-large-scale--where-toolbank-really-pays-off) in Benchmarks for real numbers at 300 tools.

Two modes cover the two ways teams actually want this to work:

| | **Discovery Mode** | **Dynamic Mode** |
|---|---|---|
| Granularity | Whole server | Individual tool |
| Tools always in context | 4 (`mcpd_find`, `mcpd_list`, `mcpd_connect`, `mcpd_get_schema`) | 2 (`find_tools`, `get_tool_schema`) |
| Best for | "Connect me to GitHub" style workflows | Cherry-picking one tool from many servers |
| After resolution | LLM talks to the server directly — ToolBank exits the data path | ToolBank lazy-connects and stays in the loop per tool call |

**v1.0.0** adds Lazy Schema Loading: Discovery Mode can hand back a stub tool list (names only, no schemas) and fetch a single tool's full schema only when it's about to be called. Real, reproducible numbers (not rough estimates) are in [Benchmarks](#benchmarks) below.

## How it works

### Discovery Mode — server-level selection

```
Step 1  LLM -> mcpd_find("github issues")
               ToolBank searches the registry
               returns: { id: "github", tools: ["create_issue", "search_repos", ...] }

Step 2  LLM -> mcpd_connect("github")
               ToolBank starts the GitHub MCP server
               returns: 20 tools now available as github__create_issue, etc.

Step 3  LLM -> github__create_issue({ title: "...", body: "..." })
               ToolBank proxies to the GitHub MCP server, returns the result
```

Real token counts for this flow are in [Benchmarks](#benchmarks) — see "Discovery Mode (before connect)" and "(after connect)".

#### Lazy Schema Loading

No proxy, no changes to the target MCP server required:

1. `mcpd_find("github")` → choose a server
2. `mcpd_connect("github", lazy_mode=true)` → get a **stub list** (tool names only, no schemas)
3. `mcpd_get_schema("github", "create_issue")` → fetch one full schema
4. `github__create_issue(...)` → direct call, as always

Pass `--sync-on-start` so the registry has schemas cached ahead of time via `toolbank-sync`.

### Dynamic Mode — tool-level selection

```
Step 1  LLM -> find_tools("create issue, post slack message")
               ToolBank searches the tool index across all servers
               returns: create_issue (github), post_message (slack)
               both tools added to tools/list

Step 2  LLM -> create_issue({ title: "Bug #42" })
               ToolBank lazy-connects to the GitHub MCP server
               executes create_issue, returns the result

Step 3  LLM -> post_message({ channel: "#eng", text: "Done" })
               ToolBank lazy-connects to the Slack MCP server
               executes post_message, returns the result
```

Real token counts for this flow are in [Benchmarks](#benchmarks) — see "Dynamic Mode (before find)" and "(after find_tools)".

## Installation

```bash
# Core — keyword search, stdio transport
pip install toolbank

# With HTTP and SSE transport (remote MCP servers)
pip install toolbank[http]

# With semantic search (sentence-transformers)
pip install toolbank[embeddings]

# With exact token counting for toolbank-benchmark (tiktoken)
pip install toolbank[benchmark]

# Full installation
pip install toolbank[all]

# Development
pip install toolbank[dev]
```

## Quick start

### 1. Build your registry

The registry is a lightweight JSON catalog of your MCP servers and their tool summaries. Build it from your MCP client's config (e.g. Cursor: `~/.cursor/mcp.json`, Claude Desktop: `~/Library/Application Support/Claude/claude_desktop_config.json`):

```bash
toolbank-sync --config /path/to/your/mcp-config.json --output registry/mcpd-registry.json
```

### 2. Point your MCP client at ToolBank

**Discovery Mode** (4 tools, connect to one server at a time):

```json
{
  "mcpServers": {
    "toolbank-server": {
      "command": "toolbank-server",
      "args": ["--registry", "/path/to/mcpd-registry.json", "--sync-on-start"]
    }
  }
}
```

**Dynamic Mode** (2 tools, cherry-pick tools across all servers):

```json
{
  "mcpServers": {
    "toolbank-gateway": {
      "command": "toolbank-gateway",
      "args": ["--registry", "/path/to/mcpd-registry.json"]
    }
  }
}
```

Or invoke the Python module directly (avoids PATH issues):

```json
{
  "mcpServers": {
    "toolbank-gateway": {
      "command": "python",
      "args": ["-m", "toolbank.dynamic.server", "--registry", "/path/to/mcpd-registry.json"]
    }
  }
}
```

### 3. Measure the token savings

```bash
toolbank-benchmark --registry registry/mcpd-registry.json
```
*(Generate your registry first with `toolbank-sync`.)* See [Benchmarks](#benchmarks) for real, reproducible numbers and what they mean.

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                        LLM / AI Client                       │
└──────────────────────┬──────────────────────────────────────┘
                       │ MCP (stdio / JSON-RPC 2.0)
          ┌────────────┴─────────────┐
          │                          │
   ┌──────▼──────┐           ┌───────▼──────┐
   │  Discovery  │           │   Dynamic    │
   │    Mode     │           │    Mode      │
   │             │           │              │
   │ mcpd_find   │           │ find_tools   │
   │ mcpd_list   │           │              │
   │ mcpd_connect│           │ LazyPool     │
   │ mcpd_get_schema│        │              │
   └──────┬──────┘           └───────┬──────┘
          │                          │
          └────────────┬─────────────┘
                       │
          ┌────────────▼─────────────┐
          │        Shared Core        │
          │                           │
          │  Registry (mcpd-registry) │
          │  KeywordSearchEngine      │
          │  ToolSearchEngine         │
          │  HybridSearch (TF-IDF +   │
          │    sentence-transformers) │
          │  ToolBankConnector           │
          │   ├─ stdio transport      │
          │   ├─ streamable-http      │
          │   └─ SSE transport        │
          └───────────────────────────┘
```

### Design principles

**A discovery layer, not a permanent proxy.** In Discovery Mode, once `mcpd_connect` resolves, the LLM gets direct tool access to the connected server. In Dynamic Mode, server connections stay lazy — a server process starts only when one of its tools is actually called.

**Offline-first registry.** Tool summaries (name, description, tags) are captured at sync time. Searches run against the cached registry with zero network traffic; full tool schemas load only on connection.

**Search degrades gracefully.** Keyword search (TF-IDF with synonyms) is the default — always available, no extra dependencies. Semantic search is optional (`pip install toolbank[embeddings]`): when installed, sentence-transformers embeddings blend with keyword results, which helps for loosely-phrased natural-language queries like *"a tool for reading web pages"* → Playwright. The keyword synonym table also understands multilingual input (e.g. Polish query terms resolve to the right English tool concepts). Keyword-first keeps installs frictionless when you don't need semantic search.

## Benchmarks

**Methodology.** Every number below comes from `toolbank-benchmark`, which instantiates the real `DiscoveryServer`/`DynamicServer` classes and measures their actual `tools/list` JSON-RPC output — not hardcoded stand-ins that can drift out of sync with the real code. Tokens are counted with [tiktoken](https://github.com/openai/tiktoken)'s `o200k_base` encoding (the GPT-4o tokenizer) when installed; without it, the CLI clearly labels its output as a rougher `char/4` estimate rather than presenting both with false equal precision. Reproduce any number here yourself:

```bash
pip install toolbank[benchmark]
toolbank-benchmark --registry registry/benchmark-registry.json --query "create github issue" --quality
```

### At realistic scale

6 servers, 61 tools, every tool has a complete real schema — [`registry/benchmark-registry.json`](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/registry/benchmark-registry.json), not a partial catalog:

| Scenario | Tools | Tokens | vs. direct load |
|---|---:|---:|---:|
| Direct (all servers, no ToolBank) | 61 | 3,786 | — |
| Discovery Mode (before connect) | 4 | 480 | 87% fewer |
| Discovery Mode (after connect: github) | 18 | 955 | 75% fewer |
| Dynamic Mode (before find) | 2 | 201 | 95% fewer |
| Dynamic Mode (after `find_tools("create github issue")`) | 7 | 390 | 90% fewer |

recall@k on this registry: **100% (8/8)** ([`quality.py`](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/toolbank/quality.py) verifies it). Read that as what it is — eight queries against sixty-one tools. It does **not** generalise to a large registry, and [On a real registry](#on-a-real-registry-448-tools-somebody-actually-wrote) below shows what happens when you try.

### At large scale — where ToolBank really pays off

This is the case ToolBank is actually built for: an organization with a large, growing catalog of MCP servers — internal tools, SaaS integrations, custom in-house APIs — where any one session only ever touches a handful. The meta-tool interface (`mcpd_find`/`mcpd_list`/`mcpd_connect`/`mcpd_get_schema`, or `find_tools`/`get_tool_schema`) costs a **fixed** number of tokens no matter how big the registry gets, while a direct load grows linearly with every tool you add. The gap only widens as you scale up. Measured on a fully-specified synthetic registry of 20 servers / 300 tools ([`registry/benchmark-registry-large.json`](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/registry/benchmark-registry-large.json)):

| Scenario | Tools | Tokens | vs. direct load |
|---|---:|---:|---:|
| Direct (all servers, no ToolBank) | 300 | 20,757 | — |
| Discovery Mode (before connect) | 4 | 480 | **98% fewer** |
| Discovery Mode (after connect: one server) | 19 | 997 | 95% fewer |
| Dynamic Mode (before find) | 2 | 201 | **99% fewer** |
| Dynamic Mode (after `find_tools(...)`) | 7 | 394 | 98% fewer |

The static meta-tool cost (480 / 201 tokens) is *identical* to the 61-tool benchmark above — that's the whole mechanism. Add a 21st server, a 500th tool, it doesn't move. Only the "Direct" column keeps growing. This is the regime — many configured MCP servers, a handful used per session — where ToolBank is the strongest option in this document.

### Small, fixed setups: skip the discovery layer

The same benchmark against a deliberately tiny registry (3 servers, 7 tools — the fixture in [`tests/conftest.py`](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/tests/conftest.py)) shows the other end of the curve, and we're showing it because it's true, not because it's flattering:

| Scenario | Tools | Tokens | vs. direct load |
|---|---:|---:|---:|
| Direct (all servers, no ToolBank) | 7 | 252 | — |
| Discovery Mode (before connect) | 4 | 480 | 90% more |
| Discovery Mode (after connect) | 7 | 576 | 129% more |
| Dynamic Mode (before find) | 2 | 201 | 20% fewer |
| Dynamic Mode (after find_tools) | 3 | 288 | 14% more |

**Rule of thumb:** if you have a small, fixed set of 2-3 MCP servers you always use, configure them directly — a discovery layer (this one or any competitor's) adds overhead you don't need. ToolBank's value curve turns sharply positive once your registry grows past a handful of servers, and keeps improving from there — see the 300-tool numbers above.

### On a real registry: 448 tools somebody actually wrote

Every registry above is either small or synthetic. Synthetic is fine for measuring the
*mechanism* — the benchmark runs the real `DiscoveryServer` and `DynamicServer` and counts their
real output — but a reader is entitled to ask whether it holds up on tools a person wrote, with
descriptions a person phrased.

So here is one. ToolBank-AutoCAD is a catalogue of MCP servers over AutoCAD: **38 categories,
448 tools**, every description and search phrasing written by hand rather than templated. It is
exactly the case named at the end of this section — a proprietary desktop application wrapped in
MCP servers — and it is large enough that loading it directly is not a thing anyone would do.

> **That repository is not public yet**, so this section does not ask you to take its existence
> on trust. The two registries derived from it are committed here —
> [`benchmark-registry-autocad.json`](registry/benchmark-registry-autocad.json) and
> [`benchmark-registry-autocad-no-intent.json`](registry/benchmark-registry-autocad-no-intent.json),
> 448 tools each, differing only in whether per-tool search phrasings survived the export. Every
> number in this section and the next was produced from those two files, and you can reproduce
> all of them from a clone of *this* repository alone. The AutoCAD source is where they came
> from, not something you need in order to check them.

Run the benchmark against the committed registry:

```bash
toolbank-benchmark --registry registry/benchmark-registry-autocad.json
```

To regenerate it from the source manifests, if you have them:

```bash
python scripts/build-registry-from-manifests.py \
    --manifests ../autocad-mcp/toolbank-manifests \
    --out registry/benchmark-registry-autocad.json
```

| Scenario | Tools | Tokens | vs. direct load |
|---|---:|---:|---:|
| Direct (all servers, no ToolBank) | 448 | 34,965 | — |
| Discovery Mode (before connect) | 4 | 480 | **99% fewer** |
| Discovery Mode (after connect: one category) | 16 | 1,241 | 96% fewer |
| Dynamic Mode (before find) | 2 | 201 | **99% fewer** |
| Dynamic Mode (after `find_tools(...)`) | 7 | 576 | 98% fewer |

Same fixed meta-tool cost — 480 and 201 tokens, identical to the 61-tool and 300-tool tables.
That is the mechanism working exactly as advertised.

### The half that token counts do not measure

Saving 99% of the tokens and then handing the model the wrong tool is not a win. It is a
regression with a good-looking chart.

So the accuracy half is measured too, on the same 448-tool registry, with
[`scripts/routing-quality.py`](scripts/routing-quality.py): sixteen plain-language requests of
the kind a person actually types — half of them in Polish, because that registry's phrasings are
bilingual — each paired with the tool that should answer it.

```bash
python scripts/routing-quality.py --registry registry/benchmark-registry-autocad.json
```

The first run was not flattering, and fixing it took two independent changes:

| Registry | keyword only | + fusion & multilingual embeddings |
|---|---:|---:|
| This repo's own benchmark registry (61 tools) | 37% | **62%** |
| ToolBank-AutoCAD, without per-tool phrasings (448 tools) | 31% | **37%** |
| ToolBank-AutoCAD, as its manifests ship today | 50% | **75%** |

Top-3, sixteen plain-language requests, half of them Polish. Top-5 on the last row is 87%.

**What the registry contributes.** The AutoCAD side requires a plain-language `Intent` list on
every one of its 448 tools — 2,387 phrasings, bilingual, enforced by its own build — and its
manifest generator was pouring them into a single category-level bag and writing them nowhere
else. Each `tools_summary` entry carried a name, a description and tags, so a discovery layer
could tell that a request was about styles but had nothing to rank `create_dimstyle` above its
twenty siblings with. That single omission is the 37% → 75% row.

The two AutoCAD rows are **the same 448 tools**, built by the same script from the same
manifests — [`benchmark-registry-autocad.json`](registry/benchmark-registry-autocad.json) and
[`benchmark-registry-autocad-no-intent.json`](registry/benchmark-registry-autocad-no-intent.json),
the latter produced with `--no-intent`. They differ in nothing but those phrasings, which is what
makes the gap evidence rather than anecdote. A discovery layer can only rank what the registry
tells it; if your tool descriptions are generated from function signatures, expect the
left-hand column.

**Phrasings have to stay phrases.** Folding them in as word-split tags scored 68% where keeping
them whole scored 75%. The hybrid engine embeds `name + description + tags` as one string and a
sentence-transformer compares *sentences*: `ile ma metrow kwadratowych to pomieszczenie` embeds
close to a user asking exactly that, while the bag `{metrow, kwadratowych, pomieszczenie}` does
not. Seven points, thrown away by tokenising too early.

### What the ranker measures, and what it does not

Everything above is **recall**: did the expected tool appear in the list at all. That is the
right thing to gate in CI, because it is a ceiling — no model can call a tool it was never
handed. It is not what a user experiences. An agent does not "get it right if the answer was in
the top 3"; it picks one tool, and if that turns out wrong, it searches again.

So the end-to-end path is measured too, with
[`scripts/rerank-quality.py`](scripts/rerank-quality.py): the candidates `find_tools` really
returned go to a frontier model, which picks one — or answers "none of these fit", writes its
own new query, and searches again. The model is never shown the expected answer and is never
told whether it was right; the second round is entered only when the model itself says nothing
fits. Numbers below are `gpt-5.6-luna`, k=10.

| Registry | ranker's own #1 | one search | + second search | recall@10 |
|---|---:|---:|---:|---:|
| This repo's benchmark registry | 25% | 87% | **87%** | 87% |
| ToolBank-AutoCAD, without per-tool phrasings | 25% | 43% | **56%** | 50% |
| ToolBank-AutoCAD, as its manifests ship today | 31% | 75% | **81%** | 93% |

Three things this says that the recall table cannot.

**The ranker's own top choice is a bad description of the system.** 31% against 87% is the
difference between reading the first line of the result list and reading the result list.

**On two of the three registries the model lands exactly on recall@10.** It extracted every
tool that was there to extract. Whatever is missing at that point is missing from the catalogue,
not from the reasoning — which is the registry-quality argument again, arrived at from the
other direction.

**Searching twice is cheap, and the model knows when to do it.** On `ile ma metrow kwadratowych
to pomieszczenie` the ranker never surfaced `get_room_data`; the model said none of these fit,
re-queried itself with `oblicz powierzchnię pomieszczenia w m²`, and found it. That recovery
costs one more round-trip:

| On the 448-tool registry | tokens | vs. loading everything |
|---|---:|---:|
| Load the whole catalogue (a plain aggregator) | 51,045 | — |
| ToolBank, before any search | 237 | 99.5% less |
| ToolBank, after one search | 1,526 | 97% less |
| ToolBank, after a second search | 3,158 | 94% less |

The extra search costs 1,632 tokens — **3.2% of what loading the catalogue costs once.** An
agent could search thirty times over and still come out ahead. A miss on the first query is a
detour, not a failure, and that is the property worth designing for: it is why "did the ranker
put it first" is the wrong question and "can the agent get there at all, cheaply" is the right
one.

**What the search contributes.** Two things were wrong here and both are fixed:

* Semantic search could not recall anything. Keyword ran first, an empty keyword result returned
  immediately, and the semantic engine's output was then filtered down to what keyword had
  already found — so embeddings could only ever *reorder* a lexical result set. Now both engines
  run independently and their ranks are combined with reciprocal rank fusion, so a tool only the
  semantic engine found can surface.
* The default embedding model was `all-MiniLM-L6-v2`, which is English-only. On a registry
  described in another language that is not a weak signal, it is no signal. The default is now
  `paraphrase-multilingual-MiniLM-L12-v2`.

One property was deliberately preserved: **keyword still decides whether anything matches at
all.** Embedding search has no concept of "no match" — it returns nearest neighbours for any
input, and the hybrid engine normalises scores so the first of them reads 1.0 even for
`xyznonexistent999`. Fusing that unfiltered destroys the empty result, which is a worse failure
than a miss: a model handed five irrelevant tools will call one, whereas an empty result is
information it can act on. So a query matching no vocabulary returns nothing; everything else
gets the full fused ranking.

> **Semantic search is an optional extra.** `pip install toolbank` gives you the keyword column.
> The right-hand column needs `pip install toolbank[embeddings]`, which pulls in
> sentence-transformers and torch. The numbers above say which is which because the difference
> is 20+ points and a reader should not have to guess which install they are reading about.

This is why [`scripts/routing-quality.py`](scripts/routing-quality.py) is in the repository and
not in a blog post: point it at *your* registry before you trust any percentage on this page,
including the good ones.

### Versus other tools on the market

One real side-by-side, run ourselves: [NCP Orchestrator](https://github.com/portel-dev/ncp) v2.3.1, installed fresh (`npx -y @portel/ncp@latest`), zero backend servers configured — its own static meta-tool interface, tokenized the exact same way:

| Tool | Meta-tools exposed | Tokens |
|---|---:|---:|
| NCP v2.3.1 (`find` + `code`) | 2 | 903 |
| ToolBank Discovery Mode (before connect) | 4 | 480 |
| ToolBank Dynamic Mode (before find) | 2 | 201 |

Measured 2026-07-29, tiktoken `o200k_base`. This is the one comparison in this section we actually ran ourselves — same tokenizer, same "before connecting anything" scenario, reproducible by anyone with Node.js installed.

For everyone else below, we're citing *published* numbers, not our own measurements — different registries, different tokenizers, different baselines. Treat these as directional, not as line-by-line comparable to the numbers above:

| Tool | Claimed reduction | Source |
|---|---|---|
| Anthropic native Tool Search (Claude Code) | ~85–96% (reported 134k→5k tokens internally) | [community writeup](https://www.opensourceforu.com/2026/01/anthropic-upgrades-open-source-mcp-to-scale-tool-rich-ai-agents/) |
| Speakeasy Dynamic Toolsets | ~99% ("100x") | [speakeasy.com](https://www.speakeasy.com/blog/how-we-reduced-token-usage-by-100x-dynamic-toolsets-v2/) |
| NCP Orchestrator (vendor-claimed) | 83–97%, varies by source | [arul.sg/ncp](https://arul.sg/ncp), [mcp.directory](https://mcp.directory/servers/ncp-mcp-orchestrator) |

**The most important line in this table isn't a percentage: Anthropic shipped this exact pattern natively into Claude Code.** If you're specifically on Claude Code, check whether you need any third-party discovery layer — this one included — before reaching for one.

### Verdict

ToolBank's savings scale with the size of your MCP ecosystem: at 300 tools across 20 servers, the measured numbers above hit 98-99%, and that curve keeps climbing the more servers you add — the meta-tool cost never grows. That's the regime this is built for: large, growing MCP deployments where dozens of servers are configured and only a handful get used per session. Against a live-tested competitor (NCP Orchestrator) it wins outright at the same task; against vendor-published numbers from Anthropic and Speakeasy it's in the same range, without an apples-to-apples test to say more than that.

**One condition on all of it.** The savings are a property of the mechanism and hold whatever
your registry contains. The *routing accuracy* is not — it is a property of your registry. The
same 448 tools, the same requests, the same frontier model end to end: **56% or 81%**, depending
on nothing but whether the catalogue carried the phrasings people search with. Run
`scripts/routing-quality.py` and `scripts/rerank-quality.py` against your own registry before
adopting this or any competitor: a router cannot rank what the catalogue does not say, and no
model can call a tool it was never handed.

**Best for:** teams with many MCP servers — SaaS integrations, internal tools, and fully custom MCP servers you build yourself. Because ToolBank works at the protocol level, it doesn't care what's behind a server: if you wrap a proprietary application in an MCP server (CAD tools, a game engine editor, an internal build system — anything you can script), ToolBank discovers and routes to it exactly like it does GitHub or Slack. The bigger and more varied that catalog gets, the more this pays off.

**Less useful for:** a handful of MCP servers you always use directly, or Claude Code users who already get equivalent behavior natively — see [Small, fixed setups](#small-fixed-setups-skip-the-discovery-layer) above.

## Registry format

The registry file (`mcpd-registry.json`) is a JSON catalog of MCP servers:

```json
{
  "mcpd_version": "1.0",
  "metadata": {
    "name": "My MCP Registry",
    "description": "Personal registry of MCP servers"
  },
  "servers": [
    {
      "id": "github",
      "name": "GitHub MCP Server",
      "description": "Official GitHub MCP server (remote). Repositories, issues, pull requests, and code search",
      "version": "remote-2025-11",
      "transport": {
        "type": "streamable-http",
        "url": "https://api.githubcopilot.com/mcp/",
        "headers": { "Authorization": "Bearer ${GITHUB_MCP_PAT}" }
      },
      "tags": ["github", "git", "code", "issues"],
      "tools_summary": [
        {
          "name": "issue_write",
          "description": "Create or update an issue or pull request",
          "tags": ["issues", "create"]
        }
      ],
      "estimated_tools_count": 90,
      "enabled": true,
      "last_synced": "2026-06-11T00:00:00Z"
    }
  ]
}
```

Full schema: [`registry/schemas/mcpd-schema.json`](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/registry/schemas/mcpd-schema.json)

## Project structure

```
toolbank/
├── toolbank/                    # Python package
│   ├── __init__.py              # Public API and version
│   ├── models.py                # Shared dataclasses
│   ├── registry.py               # Registry loader (mcpd-registry.json)
│   ├── connector.py              # MCP connector — stdio, HTTP, SSE transports
│   ├── sync.py                   # Registry builder (sync from mcp.json)
│   ├── benchmark.py              # Token savings measurement
│   ├── search/
│   │   ├── keyword_search.py    # Server-level TF-IDF search
│   │   ├── tool_search.py       # Tool-level TF-IDF search
│   │   ├── embeddings.py        # Sentence-transformer embedding engine
│   │   └── hybrid.py            # Hybrid keyword + semantic search
│   ├── discovery/
│   │   └── server.py            # Discovery Mode MCP server
│   └── dynamic/
│       ├── server.py            # Dynamic Mode MCP server
│       ├── tool_index.py        # O(1) tool lookup index
│       └── lazy_pool.py         # On-demand connection pool
├── registry/
│   ├── mcpd-registry.example.json  # Example registry
│   ├── benchmark-registry.json  # Fully-specified registry used by the Benchmarks section
│   ├── benchmark-registry-large.json  # 20-server/300-tool registry for at-scale benchmarks
│   └── schemas/
│       └── mcpd-schema.json     # JSON Schema for registry validation
├── docs/
│   ├── specification.md         # Protocol specification
│   ├── architecture.md          # Architecture deep-dive
│   ├── registry-format.md       # Registry format reference
│   └── dynamic-mcp.md           # Dynamic Mode guide
├── examples/
│   ├── cursor-config-discovery.json
│   ├── cursor-config-dynamic.json
│   └── README.md
├── tests/                       # 509 tests, 100% coverage
└── pyproject.toml
```

## Development

```bash
git clone https://github.com/KrzysztofAugiewicz/ToolBank.git
cd ToolBank
pip install -e ".[dev]"

# Run tests
pytest

# Run tests with coverage
pytest --cov=toolbank --cov-report=term-missing

# Run end-to-end integration test
python test_e2e.py

# Benchmark token savings (generate registry first with toolbank-sync)
toolbank-benchmark --registry registry/mcpd-registry.json
```

## CLI reference

| Command | Description |
|---|---|
| `toolbank-server` | Start the Discovery Mode MCP server |
| `toolbank-gateway` | Start the Dynamic Mode MCP server |
| `toolbank-sync` | Build or update the registry from an mcp.json config |
| `toolbank-benchmark` | Measure token savings for a given registry |

All commands accept `--help` for the full option reference.

## Transport support

| Transport | Install extra | Use case |
|---|---|---|
| **stdio** | _(core)_ | Local process-based MCP servers |
| **Streamable HTTP** | `toolbank[http]` | Remote HTTP MCP servers |
| **SSE** | `toolbank[http]` | Legacy remote servers (Server-Sent Events) |

Transport type is resolved automatically from the registry entry's `transport.type` field.

## Documentation

- [Protocol Specification](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/docs/specification.md) — Tool schemas, message formats, handshake sequence
- [Architecture](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/docs/architecture.md) — Data flow, shared core, design decisions
- [Registry Format](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/docs/registry-format.md) — Full registry schema reference
- [Dynamic Mode Guide](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/docs/dynamic-mcp.md) — Dynamic Mode deep-dive

## Publishing to PyPI

Releases are published automatically when a GitHub Release is created. Prerequisites:

1. Add `PYPI_API_TOKEN` to repository secrets (create at [pypi.org/manage/account/token](https://pypi.org/manage/account/token/))
2. Create a release with a tag (e.g. `v1.0.1`)

The [publish workflow](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/.github/workflows/publish.yml) builds and uploads to PyPI.

## Contributing

Contributions are welcome. Please read [CONTRIBUTING.md](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/CONTRIBUTING.md) before opening a pull request. For bug reports and feature requests, use [GitHub Issues](https://github.com/KrzysztofAugiewicz/ToolBank/issues).

## Authors

- **Krzysztof Augiewicz** — Lead Architect & Creator — [LinkedIn](https://www.linkedin.com/in/krzysztof-a-97a170185/) · [GitHub](https://github.com/KrzysztofAugiewicz)
- **Kacper Pisarczyk** — Core Contributor, Discovery & Registry Systems — [LinkedIn](https://www.linkedin.com/in/kacper-pisarczyk-b165311aa/)
- **Mateusz Wiszniowski** — Core Contributor — [LinkedIn](https://www.linkedin.com/in/mateusz-wiszniowski-263049216/) 
- **Sebastian Pawłowski** — Advisory & QA Support (testing, hardware/software provisioning) — [LinkedIn](https://www.linkedin.com/in/sebastianpawlowski/)


Full details in [AUTHORS.md](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/AUTHORS.md).

## License

[MIT](https://github.com/KrzysztofAugiewicz/ToolBank/blob/main/LICENSE)
