Metadata-Version: 2.5
Name: covenant-mcp
Version: 0.2.0
Summary: A contract linter for MCP servers — catch breaking tool-schema changes before they ship.
Project-URL: Homepage, https://github.com/Mhemd139/Covenant
Project-URL: Repository, https://github.com/Mhemd139/Covenant
Project-URL: Issues, https://github.com/Mhemd139/Covenant/issues
Project-URL: Changelog, https://github.com/Mhemd139/Covenant/releases
Author: Muhammad Masarwa
License-Expression: MIT
License-File: LICENSE
Keywords: agents,contract,drift,linter,llm,mcp,model-context-protocol,schema
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Requires-Dist: mcp<2,>=1.10
Requires-Dist: rich>=13.7
Requires-Dist: typer>=0.12
Provides-Extra: dev
Requires-Dist: anthropic>=0.40; extra == 'dev'
Requires-Dist: anyio>=4; extra == 'dev'
Requires-Dist: asyncpg>=0.29; extra == 'dev'
Requires-Dist: fastapi>=0.115; extra == 'dev'
Requires-Dist: google-genai>=1.0; extra == 'dev'
Requires-Dist: httpx>=0.27; extra == 'dev'
Requires-Dist: kopf>=1.37; extra == 'dev'
Requires-Dist: kubernetes>=29; extra == 'dev'
Requires-Dist: mypy>=1.11; extra == 'dev'
Requires-Dist: prometheus-client>=0.20; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Requires-Dist: uvicorn>=0.30; extra == 'dev'
Provides-Extra: judge
Requires-Dist: anthropic>=0.40; extra == 'judge'
Requires-Dist: google-genai>=1.0; extra == 'judge'
Provides-Extra: operator
Requires-Dist: httpx>=0.27; extra == 'operator'
Requires-Dist: kopf>=1.37; extra == 'operator'
Requires-Dist: kubernetes>=29; extra == 'operator'
Provides-Extra: proxy
Requires-Dist: fastapi>=0.115; extra == 'proxy'
Requires-Dist: httpx>=0.27; extra == 'proxy'
Requires-Dist: prometheus-client>=0.20; extra == 'proxy'
Requires-Dist: uvicorn>=0.30; extra == 'proxy'
Provides-Extra: store
Requires-Dist: asyncpg>=0.29; extra == 'store'
Description-Content-Type: text/markdown

# Covenant

**A contract linter and drift firewall for MCP servers.**

[![CI](https://github.com/Mhemd139/Covenant/actions/workflows/ci.yml/badge.svg)](https://github.com/Mhemd139/Covenant/actions/workflows/ci.yml)
![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue)
[![License: MIT](https://img.shields.io/badge/license-MIT-green)](https://github.com/Mhemd139/Covenant/tree/main/LICENSE)

![Covenant in 20 seconds: a drifted MCP tool gets quarantined, the agent gets a clean 423 instead of silently wrong data](https://raw.githubusercontent.com/Mhemd139/Covenant/main/docs/assets/covenant-film.gif)

When an MCP server changes a tool (renames an output field, tightens an input schema), nothing throws. The LLM agents depending on that tool keep calling it, read a field that no longer exists, and confidently report a wrong answer. Existing MCP scanners hash tool definitions and tell you *that* something changed; Covenant classifies every change by whether it **breaks the agent**, then enforces the verdict from CI to runtime.

Covenant makes the tool contract explicit, versioned, and enforced:

| Command | What it does |
| --- | --- |
| `covenant snapshot` | Introspect a server (stdio or HTTP) and commit its tool contracts to a deterministic `covenant.lock.json` |
| `covenant check` | Diff the live server against the baseline, classify every change **BREAKING / DEGRADED / COMPATIBLE**, exit non-zero in CI on breaking drift |
| `covenant proxy` | Transparent reverse-proxy that verifies **`tools/call` responses** against the contract and **quarantines** drifted tools: agents get a clean "tool unavailable" instead of silently wrong data |
| `MCPContract` CRD | Kubernetes operator that runs the same check on a schedule and enforces it fleet-wide |

## Quickstart

```bash
pip install covenant-mcp
covenant snapshot --server http://localhost:8000/mcp   # your server: http(s) URL or a stdio launch command
covenant check    --server http://localhost:8000/mcp   # 0 clean · 1 breaking drift · 2 config error
```

`snapshot` writes `covenant.lock.json`; commit it. Every `check` from then on diffs the live server against it. Flags can move into a committed [covenant.toml](https://github.com/Mhemd139/Covenant/blob/main/covenant.toml).

### Try the demo: break a server for real

The repo ships a real example server with a committed baseline, plus drift levers. `COVENANT_DRIFT=1` renames a live tool's output field:

```bash
git clone https://github.com/Mhemd139/Covenant && cd Covenant
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e ".[dev]"

covenant check                    # OK no drift - exit 0
COVENANT_DRIFT=1 covenant check   # catches the lie - exit 1
```

![covenant check catching a breaking change](https://raw.githubusercontent.com/Mhemd139/Covenant/main/docs/assets/drift-check.svg)

The lie is caught twice: in the declared schema (`output` rows) and in the actual response body (`behavior` rows), because the committed config probes the tool.

## The severity model

The consumer of an MCP tool is an LLM agent that re-reads tool definitions on every run, which changes what "breaking" means. Covenant classifies by one direction principle:

> **Input-side changes fail loud**: the server rejects the call, or the agent adapts → **DEGRADED** (warn; fail CI only under `--strict`).
> **Output-side changes fail silent**: the agent reads a value that is gone, retyped, or now `null`, and proceeds confidently → **BREAKING** (fail CI; quarantine at the proxy).

| Change | Tier |
| --- | --- |
| Output field removed · output required→optional · output gains `null` · structural output retype · tool removed · pinned value changed | **BREAKING** |
| Input retyped/narrowed · new required input · scalar output retype · risky enum changes · description changed | DEGRADED |
| Optional input added · output field added · input enum widened | COMPATIBLE |

Nested schemas are walked recursively (`balance.currency`, `items[].sku`). Composed schemas (`$ref`/`allOf`/`anyOf`/`oneOf`) are never guessed at; a change there flags DEGRADED for manual review. Full rationale: [Layer 0 design spec](https://github.com/Mhemd139/Covenant/blob/main/docs/specs/2026-07-01-covenant-layer0-contract-core-design.md).

## Use it in CI

Commit `covenant.toml` + `covenant.lock.json`, then:

```yaml
- name: Contract check
  run: |
    pip install covenant-mcp
    covenant check --json   # exit 1 on breaking drift, 2 on config/connection error
```

This repo runs exactly that against its own example server on every push, including a job that injects the breaking change and asserts Covenant catches it ([ci.yml](https://github.com/Mhemd139/Covenant/blob/main/.github/workflows/ci.yml)).

## Behavioral drift: probes + judge

A schema check can't see a server that *lies*: schema unchanged, response different. And most real MCP tools declare no `outputSchema` at all. Probes cover both: commit safe, **read-only** example calls in `covenant.toml`:

```toml
[[probes]]
tool = "get_transactions"
args = { account_id = "acct-001" }
```

`snapshot` stores each response's **fingerprint** (the type shape of what actually came back); `check` re-runs the probes and classifies shape drift with the same severity model.

A fingerprint remembers that *a number lives there*, not which number. When the exact value is part of the contract (a reference balance, a currency code, a unit), **pin it**:

```toml
[[probes]]
tool = "get_account"
args = { account_id = "acct-001" }
expect = { balance_usd = 4210.0, currency = "USD" }
```

`check` compares every pinned field against the live response with exact equality: no tolerance, no patterns. A mismatch is **BREAKING**. Schema and shape still match while the value lies (a balance rescaled to cents, dollars quietly converted to another currency), and that is exactly the silent failure the direction principle exists to catch. Pins are opt-in and deterministic, like `pip --require-hashes`: nothing is pinned unless you type it. Try it on the example server, where `COVENANT_SEMANTIC_DRIFT=1 covenant check` rescales the live balance ×100 and exits 1.

For drift you *didn't* pin (fields too volatile to pin, meaning shifts across the whole response), add the LLM judge:

```bash
pip install -e ".[judge]"
covenant check --judge    # [judge] model in covenant.toml: claude-* / gemini-*
```

![the LLM judge catching a semantic rescale no schema diff can see](https://raw.githubusercontent.com/Mhemd139/Covenant/main/docs/assets/judge-verdict.png)

Judge verdicts are **advisory by design**: DEGRADED, never BREAKING, because a probabilistic detector must not trigger quarantine. Details: [Layer 3 design spec](https://github.com/Mhemd139/Covenant/blob/main/docs/specs/2026-07-03-covenant-layer3-behavioral-probes-design.md).

## Runtime guard: the proxy

The linter catches drift at ship time; the proxy contains it at runtime. It forwards every JSON-RPC exchange transparently (SSE included), short-circuits a `tools/call` to a quarantined tool with a clean MCP `isError` result, and verifies the responses it forwards (below).

```bash
pip install -e ".[proxy]"
covenant proxy --upstream http://localhost:8000/mcp --port 9000
# point your MCP client at http://127.0.0.1:9000/mcp
```

| Endpoint | Purpose |
| --- | --- |
| `POST /covenant/refresh` | Re-read the baseline, re-list the upstream, re-check, update quarantine |
| `GET /covenant/status` | Currently quarantined tools and why |
| `GET /covenant/calls` | Recent call log with latency and outcomes |
| `GET /covenant/metrics` | Prometheus metrics: per-tool calls, latency, drift events, quarantine gauge |

Detection is proxy-owned: `refresh` re-lists the upstream itself, so enforcement never depends on the client's `tools/list` timing. Optional Postgres persistence keeps quarantine across restarts (`--database-url`, `[store]` extra); store writes are best-effort and never fail the request path.

### Per-call response verification

Checks are point-in-time; between two of them a server can start lying and, until v0.2.0, every call in that window sailed through. Now the proxy verifies each `tools/call` response before forwarding it, against the tool's declared `outputSchema`, else the **core fingerprint** of its committed probes (fields present in every probe response), plus any committed value pins.

A deterministic output-contract violation (reference field missing, structural retype, forbidden `null`, pinned value mismatch) is **blocked on the spot**: the agent gets a clean `isError` instead of the lie, and the tool stays quarantined until `POST /covenant/refresh`. Scalar retypes record DEGRADED and forward; extra fields pass. The verifier is deliberately bounded and fail-open, and every bypass is counted in `covenant_response_verifications_total`, never hidden: tools with no reference forward as `unverified`, bodies over a 1 MiB cap forward as `skipped_large`, and if the verifier itself breaks, the original bytes forward as `error`; the firewall never drops traffic because its own inspection failed. On SSE, only the frame carrying the matching result is held while it is verified in-process (a violation also logs a best-effort drift event before the block); progress notifications stream in real time.

Rolling out against an unfamiliar upstream? `covenant proxy --observe` detects, records, and counts, but never blocks: the classic WAF monitor-then-prevent pattern.

Try it on the example server: start the proxy, then restart the upstream with `COVENANT_SHEKEL_DRIFT=1` (amounts converted to shekels, still labeled `balance_usd`; schema and shape identical). The very next call through the proxy is blocked by the committed pin. No refresh, no `tools/list`, no CI run. Details: [per-call verification design spec](https://github.com/Mhemd139/Covenant/blob/main/docs/specs/2026-07-26-covenant-per-call-response-verification-design.md).

`docker compose up -d` also brings up Prometheus + a provisioned Grafana dashboard at `http://localhost:3000`; the quarantine stat flips green→red within one scrape of a drift:

![Grafana dashboard: blocked calls and a quarantined tool after a live drift](https://raw.githubusercontent.com/Mhemd139/Covenant/main/docs/assets/grafana-quarantine.png)

## Kubernetes: the `MCPContract` operator

Declare contract conformance instead of scripting it. The Helm chart ships the proxy, a kopf operator, and an `MCPContract` CRD. The operator re-runs the check on each contract's own schedule, writes the verdict into status, and nudges the proxy to quarantine on drift:

```bash
docker build -t covenant-mcp:0.2.0 .
helm install covenant deploy/helm/covenant --set proxy.upstream=http://my-server:8000/mcp
kubectl create configmap covenant-baseline --from-file=covenant.lock.json
kubectl apply -f examples/mcpcontract.yaml
kubectl get mcpcontracts -w        # RESULT flips clean -> breaking when the server drifts
```

A failed check is `status.result: error`, never a crash-loop. Details: [Layer 5 design spec](https://github.com/Mhemd139/Covenant/blob/main/docs/specs/2026-07-03-covenant-layer5-k8s-operator-design.md).

## Architecture

Dependency-ordered layers; each ships alone, and every enforcement surface (CI, proxy, operator) reuses the same classifier and baseline format:

| # | Layer | Extra |
| --- | --- | --- |
| 0 | Contract core: introspection, baseline, severity classifier, CLI | — |
| 1 | Transparent proxy + quarantine | `[proxy]` |
| 2 | Postgres store: durable quarantine, call log, drift events | `[store]` |
| 3 | Behavioral probes + LLM judge | `[judge]` |
| 4 | Prometheus metrics + Grafana dashboard | `[proxy]` |
| 5 | K8s operator + Helm chart | `[operator]` |

The full codebase tour lives in [docs/ARCHITECTURE.md](https://github.com/Mhemd139/Covenant/blob/main/docs/ARCHITECTURE.md); per-layer design specs (rationale, rule tables, named decisions) in [docs/specs](https://github.com/Mhemd139/Covenant/tree/main/docs/specs).

## Development

```bash
pip install -e ".[dev]"
pytest                        # Postgres-backed tests skip without COVENANT_TEST_DB
ruff check . && mypy covenant # mypy is strict
```

Layer boundaries are enforced by imports: the core depends only on `mcp`, `typer`, `rich`; everything else is an optional extra imported on use.

## License

[MIT](https://github.com/Mhemd139/Covenant/tree/main/LICENSE)
