Metadata-Version: 2.4
Name: mcp-broker
Version: 0.1.0
Summary: Local MCP broker for sharing upstream MCP servers across Codex and Claude sessions
Project-URL: Homepage, https://github.com/mcp-broker/mcp-broker
Project-URL: Documentation, https://github.com/mcp-broker/mcp-broker#readme
Project-URL: Issues, https://github.com/mcp-broker/mcp-broker/issues
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: jsonschema>=4.25.1
Requires-Dist: PyYAML>=6.0.3
Dynamic: license-file

# mcp-broker
<!-- mcp-name: io.github.navinagrawal/mcp-broker -->

`mcp-broker` is a local Model Context Protocol process broker for AI coding tools.

It lets Codex, Claude, and other MCP clients connect to one local broker entry. The broker owns upstream MCP server startup, reuse, cleanup, profile exposure, status, and safe tool routing.

The core idea is simple: do not make every agent session load every upstream tool definition before the user asks a task.

## Why this exists

AI coding sessions with many MCP servers tend to accumulate the same problems:

- every client config repeats the same MCP server list
- every new session can start duplicate upstream processes
- OAuth, browser state, local files, and database handles spread across tools
- raw tool lists consume context before the task begins
- hosted connector caches can duplicate local MCP tools
- orphaned MCP processes survive after client sessions exit

`mcp-broker` puts a small broker facade in front of those upstreams:

```text
Codex / Claude / Gemini profile
        |
        | one local MCP entry
        v
  mcp-broker-client
        |
        | Unix socket
        v
  mcp-broker-daemon
        |
        | profile gates, namespace routing, status, cleanup
        v
  upstream MCP servers
```

The client sees a small set of broker tools:

```text
broker.search_tools
broker.describe_tool
broker.call_tool
broker.status
```

The upstream MCPs still exist. They are discovered and called through the broker when a task needs them.

## Measured context reduction

On 2026-05-24, the measured Codex setup went from many raw MCP and hosted app tool definitions to one broker facade plus a pruned `codex_apps` cache.

| Surface | Before | After | Reduction |
|---|---:|---:|---:|
| Direct Codex MCP server entries | 11 | 1 | 90.91% |
| MCP tool definitions | 414 | 4 | 99.03% |
| Hosted `codex_apps` tool definitions | 195 | 39 | 80.00% |
| Combined always-loaded tool definitions | 609 | 43 | 92.94% |
| Combined serialized tool payload bytes | 1,026,171 | 185,877 | 81.89% |
| Combined `o200k_base` tool tokens | 276,989 | 45,281 | 83.65% |

The 92.94% number is a tool-definition count reduction. The 83.65% number is a token reduction for canonical serialized tool payloads measured with `tiktoken` `o200k_base`.

See [docs/context-reduction-measurement.md](docs/context-reduction-measurement.md) for evidence and caveats.

## What it does

- Runs one local broker daemon over a Unix socket.
- Exposes one lightweight stdio client shim to Codex and Claude.
- Starts upstream MCP servers on demand.
- Reuses shared upstreams across sessions when configured.
- Isolates per-session upstreams when state must not be shared.
- Maps upstream tools into stable namespaces.
- Exposes compact search, describe, call, and status tools.
- Enforces profile-level tool budgets and exposure gates.
- Blocks mutating upstream exposure unless a profile allowlist grants it.
- Stores runtime state under `$HOME/mcp/mcp-broker`, outside the repo.
- Renders Codex and Claude MCP config entries with dry-run, backup, and rollback.
- Provides LaunchAgent install and uninstall flows for macOS.
- Provides Linux systemd user-service render, install, unload, and removal flows.
- Provides Windows PowerShell Scheduled Task render, install, and removal flows.
- Includes unit, journey, live, and e2e tests through Makefile targets.

## Who this is for

Use `mcp-broker` if you:

- use Codex, Claude Code, or multiple MCP clients
- have more MCP tools than you want in every session
- need shared local MCP servers without duplicate process startup
- want one place for OAuth state, browser state, sockets, logs, and cleanup
- need per-client profiles instead of the same tool list everywhere
- want a small broker facade instead of raw upstream tool dumps

This repo is not an enterprise MCP control plane. It is local desktop infrastructure for developer-agent workflows.

## Current status

Implemented:

- YAML config loading from `config/broker.private.yaml`, created from `config/broker.example.yaml`.
- Strict YAML contract validation for runtime, clients, profiles, upstreams, and policy blocks.
- Public JSON Schema validation through `config/broker.schema.json`.
- Runtime path derivation from `runtime.root`.
- Tool namespace mapping from configured upstream prefixes.
- Local upstream subprocess lifecycle management and process-group cleanup.
- Broker daemon over Unix socket.
- MCP client shim for Codex and Claude config entries.
- Gemini exposure profile for broker facade testing.
- Dry-run client config rendering, apply-time backups, and rollback.
- LaunchAgent render and install scripts with dry-run defaults.
- Compact broker facade for search, describe, call, and status.
- Profile validation from YAML smoke probes.
- Discovery parity checks between compact client profiles.
- Public and maintainer quality gates through Makefile targets.

Wiring status:

- Codex is wired through the broker.
- Claude is wired through the broker after profile validation and manual `/mcp` acceptance.
- Gemini is a profile, not a rendered client config.

Public release status:

- The repo is designed to stay public-safe.
- Private upstream inventory, account paths, OAuth state, secrets, sockets, logs, and generated client configs stay outside git.
- PyPI, Homebrew, Docker, and registry publication are planned work. The repo now documents the intended `pipx` and Homebrew install paths, but publication is still pending.

See [ROADMAP.md](ROADMAP.md) for public-facing release work.

## Architecture

`mcp-broker` has three runtime layers:

| Layer | Responsibility |
|---|---|
| Client shim | Presents one stdio MCP server entry to each MCP client and forwards JSON-RPC over the broker socket. |
| Broker daemon | Owns profile gates, namespace routing, upstream lifecycle, status, logging, and cleanup. |
| Upstream MCP servers | Run as configured `stdio`, HTTP, streamable HTTP, or SSE connectors with shared or per-session process policy. |

The config file is the contract. Profiles decide exposure, upstreams define transport and lifecycle behavior, and smoke probes define safe read calls for validation.

## Comparison

| Approach | Best fit | Tradeoff |
|---|---|---|
| Raw MCP client config | Small setups with a few tools. | Every session loads the full tool list and each client repeats config. |
| Simple MCP proxy | Forwarding one server to one client. | Does not own upstream lifecycle, profile budgets, or cross-client cleanup. |
| Hosted app connectors | SaaS tools managed by the client provider. | Local MCP state and cross-client parity remain outside user control. |
| `mcp-broker` | Local developers with many upstream MCPs across Codex, Claude, and other clients. | Adds a local daemon and config contract that must be installed and monitored. |

## Screenshots Or GIF

The public release should include a short terminal GIF showing:

```text
make config-init
make config-validate
make broker-status
make codex-facade-smoke
```

It should also show the MCP client `/mcp` view with one `mcp-broker` entry and the broker `status` tool listing profile-visible upstream state.

## Quickstart

Prerequisites:

- macOS with `launchctl` for LaunchAgent use.
- Python 3.10 or newer available as `python3`.
- `make`.
- Node.js and `npx` for npm-based upstream MCP servers.
- A clone of this repo.

Intended package installs after publication:

```bash
pipx install mcp-broker
brew install mcp-broker
```

Homebrew should install the same console scripts as the Python package. Package
install must not write MCP client config; client wiring stays an explicit
Makefile action.

Create the local venv, install dependencies, and verify runtime layout:

```bash
make setup
```

Create private config from the public template:

```bash
make config-init
```

`config-init` creates the destination directory when needed and copies the public
template as the starting point. It does not import local MCP inventory, user
paths, or secrets.

Edit `config/broker.private.yaml` for local upstreams. Keep secret values out of config. Use environment variable names or files under:

```text
$HOME/mcp/mcp-broker/secrets/
```

Run the quality gate:

```bash
make quality-gate
```

Validate the configured YAML contract:

```bash
make config-validate
```

Start the broker:

```bash
make broker-start
```

Check status:

```bash
make broker-status
```

For the full install flow, see [docs/install.md](docs/install.md).

## Runtime layout

Default runtime root:

```text
$HOME/mcp/mcp-broker/
|- backups/
|- logs/
|- renders/
|- run/
|- secrets/
|- sockets/
`- state/
   `- upstreams/
```

Runtime files are not repo files. Upstream OAuth state, browser state, secret files, sockets, logs, rendered client configs, backups, and daemon state belong under the runtime root.

See [docs/runtime-layout.md](docs/runtime-layout.md).

## Client wiring

Back up a client config:

```bash
make config-backup CLIENT=codex
```

Dry-run render:

```bash
make config-render CLIENT=codex CONFIG_RENDER_APPLY=0
```

Apply after reviewing the rendered file under `$HOME/mcp/mcp-broker/renders/`:

```bash
make config-render CLIENT=codex CONFIG_RENDER_APPLY=1
```

Rollback:

```bash
make config-rollback CLIENT=codex
```

Use `CLIENT=claude` after the Claude profile smoke passes and Claude wiring is intended.

## Compact broker facade

The compact facade keeps chat-facing profiles small:

| Tool | Purpose |
|---|---|
| `broker.search_tools` | Search configured upstream tools by query. |
| `broker.describe_tool` | Return schema and metadata for one upstream tool. |
| `broker.call_tool` | Call one upstream tool through broker routing. |
| `broker.status` | Show profile-visible upstream state, passive auth probes, and last errors without starting tools. |

Codex `/mcp` shows the single `mcp-broker` entry by design. Per-upstream visibility comes from `broker.status`.

## Profiles and safety

Profiles decide which upstreams a client can see and call.

Supported concepts:

- `max_tools` protects clients from huge tool lists.
- `compact_tools_enabled` exposes broker facade tools instead of raw upstream tools.
- `allow_mutating_upstreams` is required before a mutating upstream can be exposed.
- `shared` mode reuses one upstream process where shared account state is acceptable.
- `per_session` mode isolates upstream state per client session.
- `disabled` mode keeps compatibility records without exposing the upstream.

Protected surfaces such as OAuth, browser state, filesystem roots, and databases require explicit config and validation. Public examples stay disabled or placeholder-based.

See [docs/security-review.md](docs/security-review.md) and [docs/upstream-compatibility-matrix.md](docs/upstream-compatibility-matrix.md).
For a deeper safety checklist, see [docs/safety.md](docs/safety.md).

## Config contract

The public template is [config/broker.example.yaml](config/broker.example.yaml).
The matching JSON Schema is [config/broker.schema.json](config/broker.schema.json).

Supported top-level sections:

```yaml
schema_version: 1
runtime: {}
broker: {}
profiles: {}
clients: {}
upstreams: {}
```

The loader rejects unknown keys. Runtime placeholders such as `{runtime.root}`, `{runtime.state_dir}`, and `{runtime.secrets_dir}` can be used in upstream command, args, working directory, and env file paths.

`make config-validate` checks the selected `CONFIG_PATH` against the public JSON Schema first, then runs the runtime loader so semantic rules are enforced from the same code path the broker uses.

Each enabled upstream exposed to a profile should define a safe smoke probe:

```yaml
smoke:
  query: read example graph
  tool: example-store.read_graph
  arguments: {}
  call: true
```

`make profile-validation PROFILE=<profile>` validates every enabled upstream visible to that profile through `broker.status`, `broker.search_tools`, `broker.describe_tool`, and the configured safe `broker.call_tool`.

## Codex operator acceptance

Repo-owned tests validate broker behavior through the local client shim. The last
Codex-specific check has to run inside an active Codex session because that is
where the deferred MCP wrapper tools exist.

Generate the current acceptance steps from YAML:

```bash
make codex-deferred-acceptance
```

The target reads the configured `smoke` probes and prints the exact
`mcp__mcp_broker__` wrapper calls for search, describe, and safe call. It does
not invoke Codex, does not call an external LLM session, and is not part of
`make quality-gate`.

See [docs/codex-deferred-tool-acceptance.md](docs/codex-deferred-tool-acceptance.md).

## LaunchAgent

Render without writing:

```bash
make launchagent-install
```

Apply and load:

```bash
make launchagent-install LAUNCHAGENT_APPLY=1
make launchagent-load
make broker-status
```

Unload or remove:

```bash
make launchagent-unload
make launchagent-uninstall LAUNCHAGENT_APPLY=1
```

## systemd

Linux user-service install uses the same runtime root and config path contract:

```bash
make systemd-install
make systemd-install SYSTEMD_APPLY=1
make systemd-load
```

For package installs, set `MCP_BROKER_DAEMON_COMMAND` to the installed daemon
path before applying the service.

## Windows

Windows startup uses PowerShell Scheduled Task commands with the same runtime
root and config path contract:

```bash
make windows-install
make windows-install WINDOWS_APPLY=1
make windows-load
```

Remove it with:

```bash
make windows-unload
make windows-uninstall WINDOWS_APPLY=1
```

## Test and release gates

Run all test tiers:

```bash
make test
```

Run the public quality gate:

```bash
make quality-gate
```

The coverage gate uses line and branch coverage for Python source.

Run smoke and runtime cleanup checks:

```bash
make config-validate
make broker-smoke
make broker-stop
make broker-reap
make doctor
make release-smoke
```

Release or client config apply should wait for:

- `make quality-gate`
- `make config-validate`
- `make broker-smoke`
- dry-run config render for each intended client
- rollback test
- `make release-smoke`
- `make doctor` with no stale broker-owned resources

See [docs/release-checklist.md](docs/release-checklist.md).

## Public commands

These targets use this repo plus declared Python and Node prerequisites:

```bash
make setup
make config-init
make test
make test-unit
make test-journey
make test-live
make test-e2e
make test-cov
make precommit
make quality-gate
make config-validate
make broker-smoke
make broker-start
make broker-status
make broker-stop
make broker-reap
make doctor
make config-backup
make codex-app-policy
make config-render
make config-rollback
make tools-count
make facade-smoke
make codex-facade-smoke
make claude-facade-smoke
make gemini-facade-smoke
make profile-validation
make codex-profile-validation
make claude-profile-validation
make gemini-profile-validation
make discovery-parity
make codex-claude-discovery-parity
make codex-deferred-acceptance
make launchagent-install
make launchagent-load
make launchagent-unload
make launchagent-uninstall
make systemd-install
make systemd-load
make systemd-unload
make systemd-uninstall
make windows-install
make windows-load
make windows-unload
make windows-uninstall
make linux-container-smoke
make windows-powershell-smoke
make release-smoke
make public-release-dry-run
```

`make quality-gate` is repo-local. It does not call personal scripts outside this repo.

`make codex-deferred-acceptance` is maintainer-only. It does not invoke Codex
or an external LLM session. It reads the same YAML `smoke` probes and prints the
exact `mcp__mcp_broker__` deferred wrapper calls to run inside an active Codex
session. See [docs/codex-deferred-tool-acceptance.md](docs/codex-deferred-tool-acceptance.md).

Maintainer-only aliases may exist for this machine's private quality system. They are guarded by checks for `~/.llm-shared/scripts` and are not part of the public setup path.

## Project tree

```text
mcp-broker/
|- Makefile
|- README.md
|- pyproject.toml
|- requirements.txt
|- config/
|  |- broker.example.yaml
|  `- broker.schema.json
|  `- broker.private.yaml        # local, ignored by git
|- docs/
|- registry/
|- scripts/
|- src/
|  `- mcp_broker/
|- tests/
|  |- unit/
|  |- journey/
|  |- live/
|  |- e2e/
|  `- support/
`- var/                         # tracked skeleton; generated contents ignored
```

Generated reports stay under `var/`, especially `var/coverage/`, `var/test-logs/`, and `var/quality/`.

## Docs

- [SECURITY.md](SECURITY.md)
- [CONTRIBUTING.md](CONTRIBUTING.md)
- [ROADMAP.md](ROADMAP.md)
- [docs/install.md](docs/install.md)
- [docs/migration.md](docs/migration.md)
- [docs/adoption-guide.md](docs/adoption-guide.md)
- [docs/comparison.md](docs/comparison.md)
- [docs/distribution.md](docs/distribution.md)
- [docs/github-publication.md](docs/github-publication.md)
- [docs/community-launch.md](docs/community-launch.md)
- [docs/auth-recipes.md](docs/auth-recipes.md)
- [docs/architecture.md](docs/architecture.md)
- [docs/protocol.md](docs/protocol.md)
- [docs/runtime-layout.md](docs/runtime-layout.md)
- [docs/safety.md](docs/safety.md)
- [docs/security-review.md](docs/security-review.md)
- [docs/upstream-compatibility-matrix.md](docs/upstream-compatibility-matrix.md)
- [docs/codex-deferred-tool-acceptance.md](docs/codex-deferred-tool-acceptance.md)
- [docs/context-reduction-measurement.md](docs/context-reduction-measurement.md)
- [docs/release-checklist.md](docs/release-checklist.md)
- [docs/troubleshooting.md](docs/troubleshooting.md)

## Design rules

- Keep upstream definitions in central config.
- Keep runtime state under `$HOME/mcp/mcp-broker`.
- Keep private upstream inventory in `config/broker.private.yaml`, which is ignored by git.
- Keep secret values out of config and source.
- Do not hardcode personal paths in source, tests, docs, or public config.
- Run build, test, runtime, and config operations through the Makefile.
