Metadata-Version: 2.5
Name: pytest-mcp-honesty
Version: 0.1.0
Summary: Bidirectional contract testing for FastMCP servers: catch schemas that promise less than the implementation demands, and implementations that enforce less than the schema promises.
Project-URL: Repository, https://github.com/senor14/pytest-mcp-honesty
Author: senor14
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: contract-testing,fastmcp,json-schema,mcp,pytest
Requires-Python: >=3.10
Requires-Dist: fastmcp>=2.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Description-Content-Type: text/markdown

# pytest-mcp-honesty

**Bidirectional contract testing for FastMCP servers, in your own pytest suite.**

A tool's schema is a contract, and servers break it in both directions:

- **The schema promises less than the implementation demands.** A param is
  `str | None = None` in the signature — so the generated schema calls it optional — but the
  handler errors without it. Every agent that trusts the schema burns a failed call to learn
  the truth. (Found in the wild: a server whose `resolve_team` declared `required: []` and then
  errored "provide club_name or org_id".)
- **The implementation enforces less than the schema promises.** Required params can be
  omitted, or structurally wrong values are accepted, and garbage flows into your handler.

`check_contracts` probes every tool over FastMCP's **in-memory client** — no network, no auth,
hermetic — so a failed probe is a finding about your contract, not your environment:

```python
from pytest_mcp_honesty import check_contracts
from my_app import server  # your FastMCP instance

async def test_tool_contracts():
    report = await check_contracts(server)
    assert not report.violations, report.summary()
```

Three probe families, all deterministic (same schema → same probe):

| Probe | Catches |
|---|---|
| minimal valid call (declared-required args only) | schemas that under-declare what the tool needs |
| drop one required arg | schemas that over-declare, or missing validation |
| structurally wrong value (uncoercible — a dict where a scalar belongs) | declared types that are not enforced |

The wrong-value probe deliberately avoids values Pydantic's lax coercion legally repairs
(`"5"` → `5` is not a violation); it sends values nothing can coerce.

## What this is not

- Not protocol conformance — the transport/spec layer is a different job (see the official
  MCP Inspector, or mcp-java-testkit on the JVM).
- Not a fuzzer — probes are deterministic transcriptions of the contract, not random inputs
  (see mcp-fuzzer for security-style fuzzing).
- Not schema *portability* — whether your schema survives specific clients is
  [mcp-portability-lint](https://github.com/senor14/mcp-portability-lint)'s job; this checks
  whether your schema tells the truth about your own server.

## Development

Maintained by one person with AI assistance; where AI tooling contributed is logged per commit
in [docs/ai-maintenance-log.md](docs/ai-maintenance-log.md).

## License

Apache-2.0
