boost / docs / mcp-hub · RAG Phases 1–3

RAG · shipped in PR #23 · #30 · #64

Semantic skill search, surfaced as a hub.

boost turns tapped skill repos into a searchable corpus and lets an agent ask for "the best skill for <task>." Three phases got it there: Phase 1 — full-content BM25 retrieval with an LLM rerank; Phase 2 — an opt-in dense embeddings backend; Phase 3 — an extensible MCP registry that exposes all of it. Everything degrades gracefully and the default install stays zero-dependency.

3RAG phases shipped

6MCP tools on the registry

89.6%Mutation kill on core/

0Deps in the default install

The whole stack

Three phases, one degrade-safe path

Each phase is a layer that improves retrieval without ever being required. If the dense backend isn't installed, search uses BM25; if there's no index yet, it uses frontmatter; if there's no LLM, it skips the rerank. Nothing hard-fails.

Fig 1 — The layers are a preference order, not a pipeline: retrieval picks the highest layer that's actually available and falls through on absence.

Indexing · boost reindex

From cloned repos to two stores

Tapping a repo already clones its skills to local disk, so building the index is a local batch job — no extra network for BM25. Bodies are stripped of frontmatter, chunked, and written to a persisted, commit-keyed index; the opt-in dense store additionally embeds each chunk.

Fig 2 — boost reindex builds the BM25 store; boost reindex --dense additionally embeds each chunk. Both key on the git commit already recorded per tap, so only changed taps are re-read.

Query · rag.search()

Retrieve, then rerank

A search is two stages. First _retrieve_any pulls a wide shortlist (max(60, limit×4)) from the best available engine; then, if an LLM is present, it reorders the top of that shortlist by task fit — otherwise it keeps the retrieval order.

Fig 3 — search() returns None only when no index exists, which is the single signal the MCP boost_search tool uses to fall back to frontmatter search — so a brand-new install still answers.

The hub · architecture

One registry, many handlers

The JSON-RPC server never knows what tools exist — it asks the Registry in core/mcp.py. Handlers that reach out to the world (like GitHub) probe for their dependency and degrade to a message instead of crashing.

Fig 4 — The server holds no tool knowledge; it delegates every tools/list / tools/call to the registry. New tools slot in on the right without touching the loop on the left. boost_search is the handler that calls rag.search() from Fig 3.

The hub · sequence

A `boost_discover_github` call, end to end

The happy path reaches GitHub; the shaded branch is what happens when gh is missing or the search fails — the server answers, never dies.

Fig 5 — Steps 5–6 are the only ones that touch the network. Everything below step 4 has a mirror in the shaded alt branch, so a missing dependency is a message, not an exception.

The refactor

From an if/elif ladder to a registry

Same behaviour, same wire protocol — but adding tool #7 no longer means editing the dispatcher, and an unknown tool resolves in exactly one place.

before — configuration.py

# flat dispatch: every tool is a branch
def _mcp_tool(tool, args):
    if tool == "boost_search":
        ...
    if tool == "boost_list":
        ...
    # +1 tool = +1 branch here
    return None, False

_MCP_TOOLS = [ {…5 hand-kept specs…} ]

after — core/mcp.py + configuration.py

# the seam: an ordered name → (spec, handler) map
REGISTRY = mcp.Registry()

REGISTRY.register(
    "boost_discover_github",
    "Discover new SKILL.md repos…",
    {"type": "object", "properties": {…}},
    _tool_discover_github)          # ← the only edit

_MCP_TOOLS = REGISTRY.specs()       # list stays in sync

Cheat sheet

Explain it two ways

Every concept, twice: the simple version on the left, the deep technical on the right. Open both and compare — they describe the same thing at different altitudes.

🧸 Simple

🔬 Deep technical

Retrieval engine · Phases 1–2

Cut pages into index cards chunking

A whole skill page is too big to match on well. So we cut it into small overlapping index cards. Overlap means a sentence that lands on the seam still shows up whole on one card, so we never lose it.

Passage-level chunking rag.chunk()

read_body pulls each item's Markdown from ~/.boost/repos and strips frontmatter; chunk() slices it into ~1000-char windows with 150-char overlap, capped at 40 chunks per item. Retrieval is therefore passage-level: a match on one section surfaces the whole skill, scored by its best chunk.

Which words really matter BM25

To rank cards, we count shared words — but rare words ("jest") count more than common ones ("the"), and a long card doesn't win just for being long. That's the always-on ranker; it needs no internet and no AI.

Full-content BM25 k1=1.2 · b=0.75

Phase 1: a pure-stdlib sparse ranker over full bodies (K1=1.2, B=0.75), scoring by term frequency × inverse document frequency with length normalization. Per (name, tap) the max chunk score wins. The postings live in a persisted rag_index.json with the tuning params stored inline, so they're tunable without a format change.

Big handful, then a wise sort two-stage

First grab a generous handful of maybe-matches fast. Then, if a smart helper (an AI) is around, ask it to put the very best on top for your exact task. If it's not around, the fast ranking is already pretty good — we just use that.

Retrieve → LLM rerank rag.search()

Stage 1: _retrieve_any returns a wide shortlist of max(60, limit×4). Stage 2: rerank() asks the core/ai.py bridge (local claude CLI → ANTHROPIC_API_KEY) to reorder names best-first, tagging the result "Claude relevance"; with no LLM it returns the retrieval order tagged "BM25 full-content".

Search by meaning, not words dense

An optional upgrade turns every skill into a point in "meaning space," so "test my React app" can find a jest skill even with no shared words — it just finds the nearest points. It's optional because it needs an extra install and a key.

Dense embeddings + sqlite-vec Phase 2 · [rag]

Chunks are embedded via embed over urllib — preferring VOYAGE_API_KEY (voyage-3, 1024-d) then OPENAI_API_KEY (text-embedding-3-small, 1536-d) — and stored in a sqlite-vec vec0(… distance_metric=cosine) table. Retrieval does a KNN over-fetch (max(k×8, 200)) then reduces per-entry. A provider/dimension change invalidates the store so stale vectors are never served.

Only re-read what changed incremental

When you refresh, we don't re-read every shelf — only the ones whose books actually changed since last time. That keeps refreshes quick even with a big library.

Commit-keyed reuse boost reindex

Both stores key on the git commit already recorded per tap. On rebuild, unchanged taps are reused verbatim and only changed taps' rows are replaced (reused / reindexed in the stats). Bumping INDEX_VERSION wipes and rebuilds. Because taps are already cloned to disk, BM25 indexing needs no network.

The hub · Phase 3

The toy box registry

Imagine a toy box with labelled slots. Each toy knows its own name and what it does, and it puts itself into a slot. When someone asks "what toys do you have?" you just read the labels. When they say "play with the drum," you grab the drum from its slot. You never keep a giant list in your head — the box does.

Registry: ordered name → (spec, handler) core/mcp.py

Registry holds three parallel structures keyed by tool name: an insertion-ordered name list, a specs map ({name, description, inputSchema}), and a handlers map. register() rejects empty or duplicate names; specs() returns the JSON payload in registration order; call(name, args) looks up the handler and returns (None, False) for anything unknown. It's pure, stdlib-only, and mutation-gated.

What each toy promises handler

Every toy promises the same thing: "give me your ask, and I'll hand back some words, plus a thumbs-up or thumbs-down." That's it. Because they all promise the exact same shape, the toy box doesn't care which toy it grabbed — they're interchangeable.

The handler contract (text, is_error)

A handler is fn(args: dict) -> (text, is_error). text is the human-readable result; is_error becomes the MCP result.isError flag. A handler may return text is None to mean "no result," which the server renders identically to an unknown tool — so the -32602 path lives in exactly one place. Handlers close over the core engine (rag, catalog, store) but the registry knows nothing about them.

The scout that looks outside boost_discover_github

Most toys play with things already in the room. This one is a scout: it opens the door and looks down the street (GitHub) for new skills to bring home. If the door is locked (no gh tool installed), it doesn't panic — it just says "hey, you need a key, here's how to get one" and comes back inside.

First reach-out tool gh code search

_tool_discover_github probes shutil.which("gh"), then calls discovery.github_skill_search(query, limit) — a single gh api search/code?q=filename:SKILL.md… page (no cache write, limit clamped to 100, query URL-encoded and appended to the filename filter). Results map to {repo, path, url, description} and render one per line. Every failure mode (TimeoutExpired, OSError, non-zero exit, bad JSON) returns None so the handler emits a hint instead of raising.

Never cry, always answer degradation

The rule for any toy that reaches outside: never throw a tantrum. If something it needs is missing, it must still say something friendly and useful. That way the whole toy box keeps working even when one toy can't reach its toy.

Probe-then-degrade core/ai.py pattern

Mirrors the established core/ai.py discipline: detect the dependency, and if absent return a short actionable message rather than raising. The MCP server is a long-lived stdio loop — an uncaught exception in one tools/call would be caught by the server's blanket handler, but degrading inside the tool gives a precise, testable message and keeps is_error semantically correct.

Old handles still turn shims

We rearranged the room, but we kept the old light switches on the wall so anything that used to flip them still works. Nobody has to relearn where the switches are.

Back-compat: _MCP_TOOLS / _mcp_tool no churn

_MCP_TOOLS = REGISTRY.specs() and a one-line _mcp_tool() that delegates to REGISTRY.call() preserve the two symbols the server loop and the existing test suite reference. The refactor is behaviour-preserving: the tools/list order is unchanged for the first five tools, and only the new sixth entry is appended — so TestMcp needed a single one-line assertion update.

Extension contract

Adding tool #7

The whole point of the seam: a new capability — a database query, another external API — is a handler plus one register(). No server edit, no dispatcher edit.

your_new_tool.py

def _tool_x(args):                       # fn(args) -> (text, is_error)
    if not _dependency_available():
        return "needs <dep> — here's how to get it", True   # degrade
    return _do_the_work(args), False

REGISTRY.register(
    "boost_x",
    "one-line description shown to the agent",
    {"type": "object", "properties": {…}},
    _tool_x)                              # tools/list + tools/call: automatic

Rule	Why
Handler returns `(text, is_error)`	Uniform shape lets the registry stay tool-agnostic.
Reach-out tools must degrade, never raise	The stdio server is long-lived; a missing `gh`, offline network, or absent `[rag]` extra must not kill it.
Registration order = `tools/list` order	Deterministic surface; existing clients see a stable ordering with new tools appended.
Put reusable logic in `core/`	It gets unit-tested and mutation-gated — the registry itself lives in `core/mcp.py` for exactly this reason.