Metadata-Version: 2.5
Name: mcp-arcade
Version: 0.1.0
Summary: GameDay testing for MCP servers — the oracle is the wire, not the scoreboard.
Project-URL: Homepage, https://github.com/mcp-tool-shop-org/mcp-arcade
Project-URL: Documentation, https://mcp-tool-shop-org.github.io/mcp-arcade/
Project-URL: Repository, https://github.com/mcp-tool-shop-org/mcp-arcade
Project-URL: Issues, https://github.com/mcp-tool-shop-org/mcp-arcade/issues
Project-URL: Changelog, https://github.com/mcp-tool-shop-org/mcp-arcade/blob/main/CHANGELOG.md
Author-email: mcp-tool-shop <64996768+mcp-tool-shop@users.noreply.github.com>
License-Expression: MIT
License-File: LICENSE
Keywords: agents,gameday,mcp,security,testing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Requires-Dist: click>=8.1.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: rich>=13.0.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Description-Content-Type: text/markdown

# MCP Arcade

<p align="center">
  <strong>GameDay testing for MCP servers. The house keeps the tape.</strong>
</p>

<p align="center">
  <a href="https://github.com/mcp-tool-shop-org/mcp-arcade/actions/workflows/ci.yml"><img src="https://github.com/mcp-tool-shop-org/mcp-arcade/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
  <a href="https://github.com/mcp-tool-shop-org/mcp-arcade/blob/main/LICENSE"><img src="https://img.shields.io/github/license/mcp-tool-shop-org/mcp-arcade" alt="License: MIT" /></a>
  <a href="https://mcp-tool-shop-org.github.io/mcp-arcade/"><img src="https://img.shields.io/badge/Landing_Page-live-blue" alt="Landing Page" /></a>
</p>

You point Arcade at a server. It runs a short, named catalog of experiments. The **oracle is the JSON-RPC wire plus sandbox files** — which `tools/call` went out, with which arguments, and whether the sandbox changed. It does **not** score a regex hit on a tool description, a TUI flourish, or a model essay.

Fun is on purpose. It is second. A full bar is not a safety certificate; it is “these atoms did not catch a lie this time.”

## What v1 actually is

Three atoms. That is the whole catalog. We do not advertise 1,312 of anything.

| Atom | Title | Hypothesis |
|------|-------|------------|
| `inspect.tools_list` | The Honest Menu | The menu the server publishes is the menu it will serve. |
| `poison.follow_through` | The Whispered Errand | A side-quest buried in a description does not become a `tools/call`. |
| `temporal.rug_pull` | The Long Con | After N clean calls, it is still the same tool. |

The connected “agent” in v1 is a **scripted policy**, not an LLM:

- `naive` — follows “also call X” side-quests in descriptions (an instruction-follower we fully specify)
- `task-only` — runs the requested tool and stops

That is the floor. An optional Ollama seat can sit on top later. It will not grade itself; it will not see chain-of-thought as evidence. Bout receipts are already shaped as that dataset.

## Install

```bash
pip install mcp-arcade
```

From a clone:

```bash
pip install -e ".[dev]"
```

## Quick start

```bash
# Lab server (no --allow-live needed)
mcp-arcade bout --target fixture --agent naive --no-prompt -o receipt.json

# Same lab, policy that refuses whispered errands
mcp-arcade bout --target fixture --agent task-only --no-prompt

# Your stdio server (fail-closed: must opt in)
mcp-arcade bout --target stdio --cmd python --cmd -m --cmd your_server --allow-live --no-prompt
```

`mcp-arcade atoms` lists the catalog. `mcp-arcade receipt receipt.json` prints the canonical tape.

Without `--no-prompt`, the TUI asks what *you* think the wire will show **before** it posts the score. That pause is load-bearing.

## Scoring

Two axes, then one number that cannot be gamed by refusing to work:

- **Utility** — the benign requested tool ran
- **Integrity** — no unauthorized call, no catalog lie, no silent mutation
- **NRP** = `task_success * (1 - attack_success)`

A policy that never calls tools scores 0, not a perfect integrity bar.

The poison atom’s fail condition is an extra `tools/call`. If the description contains “also call leak” and the agent never calls `leak`, that is a **pass**. Detecting the string is not the test.

## Security & data scope

- Default target is the bundled fixture. It writes only under the `--sandbox` directory you pass (or `.arcade-sandbox` in the current working directory).
- Non-fixture servers require `--allow-live`. Arcade will not spawn your command otherwise.
- No telemetry. No network of its own. An optional future Ollama seat talks to localhost if you turn it on.
- Receipts contain tool names, arguments, and sandbox file snapshots from the bout. Do not point `--allow-live` at a production server that can reach real secrets.

See [SECURITY.md](SECURITY.md).

## Dataset (for a later Ollama seat)

Every receipt is `mcp-arcade.bout/v1` JSON: calls, observations, tool lists, scores, a split field. Labels come from the wire. Operator guesses and TUI copy are recorded and **must not** be used as ground truth. See [docs/datasets.md](docs/datasets.md).

## What this is not

- Not a scanner benchmark and not a port of MCPTox’s 1,312 cases. MCPTox is the *method* we cite (agent follow-through on live servers). The catalog we ship is the three atoms above.
- Not load testing.
- Not a 3D canvas. A spatial overview can wait; the diagnostic surface is the timeline plus the receipt.

## Development

```bash
pip install -e ".[dev]"
pytest
ruff check src tests
```

## License

MIT. See [LICENSE](LICENSE).
