Metadata-Version: 2.5
Name: agent-capmap
Version: 0.1.0
Summary: Inventory the MCP tools reachable by AI agents and flag dangerous capability combinations with policy-as-code.
Project-URL: Homepage, https://github.com/NKwolv/agent-capmap
Project-URL: Documentation, https://github.com/NKwolv/agent-capmap#readme
Project-URL: Issues, https://github.com/NKwolv/agent-capmap/issues
Project-URL: Changelog, https://github.com/NKwolv/agent-capmap/blob/main/CHANGELOG.md
Project-URL: Security, https://github.com/NKwolv/agent-capmap/security/policy
Author: NKwolv
License-Expression: MIT
License-File: LICENSE
Keywords: ai-agents,mcp,model-context-protocol,sarif,security
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Requires-Python: >=3.11
Requires-Dist: anthropic>=1.11
Requires-Dist: certifi>=2026.10.19
Requires-Dist: httpx>=0.27
Requires-Dist: jsonschema>=4.20
Requires-Dist: pydantic>=2.7
Requires-Dist: pyyaml>=6.0.2
Provides-Extra: dev
Requires-Dist: bandit>=1.8; extra == 'dev'
Requires-Dist: cyclonedx-bom>=7.5; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pip-audit>=2.7; extra == 'dev'
Requires-Dist: pre-commit>=3.7; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Requires-Dist: types-jsonschema; extra == 'dev'
Requires-Dist: types-pyyaml; extra == 'dev'
Requires-Dist: uv>=0.13.4; extra == 'dev'
Requires-Dist: virtualenv>=21.14.9; extra == 'dev'
Requires-Dist: zizmor>=1.0; extra == 'dev'
Description-Content-Type: text/markdown

# agent-capmap

**Find out what your AI agents can actually do, and which combinations of their tools
an attacker could abuse.**

AI assistants such as Claude Desktop, Cursor, VS Code and Claude Code get their powers
from MCP servers: read files, browse the web, send email, run shell commands. Each tool
can look harmless on its own. The danger is usually the combination: an agent that can
read private data, read attacker-written content (a web page, an issue, an email), and
send data out can be talked into leaking that data. That is the "lethal trifecta".

agent-capmap inventories every MCP server and tool reachable by each agent, classifies
what each tool can do, and applies policy-as-code rules to the **combination** of tools
per agent. Findings come with evidence and can gate CI through SARIF and exit codes.

> **Status: pre-release (0.1.0).** All features below work. The classifier's accuracy
> has not been benchmarked yet; see [Accuracy](#accuracy).

## Install

Requires Python 3.11+.

```sh
pipx install agent-capmap        # or: uv tool install agent-capmap
```

Releases are built and published by GitHub Actions with PyPI trusted publishing, so
each file on PyPI carries an attestation linking it to the workflow run that built it.

For a fully hash-pinned install, with the exact dependency versions CI tests, install
from a clone instead:

```sh
git clone https://github.com/NKwolv/agent-capmap
cd agent-capmap
python -m venv .venv
. .venv/bin/activate            # Windows PowerShell: .venv\Scripts\Activate.ps1
pip install --require-hashes -r requirements/runtime.lock
pip install --no-deps .
```

> **Locked-down Windows machines:** if your organization's policy (Device Guard,
> AppLocker, Smart App Control) blocks `agentcapmap.exe` or `pipx.exe`, run the tool
> through Python instead, which is the same program. If you installed with
> `pip install agent-capmap`, run `python -m agentcapmap scan --user`. If you installed
> with pipx, the package lives in pipx's own environment, so plain `python` will not
> find it: use that environment's interpreter (`python -m pipx environment --value
> PIPX_LOCAL_VENVS` shows the folder):
> `<venvs>\agent-capmap\Scripts\python.exe -m agentcapmap scan --user`. Do not try to
> bypass the policy; ask your administrator to allow the launcher if you want it.

> **Windows:** if `pip install` fails with "No such file or directory" deep inside
> `site-packages`, the path is too long for Windows' 260-character limit. Clone to a
> short path (for example `C:\src\agent-capmap`) or enable long path support.

## Quickstart (5 minutes, no API key)

```sh
# 1. See findings on a ready-made inventory of five example agents
#    (from a clone, use examples/demo/inventory.json directly)
curl -fsSLO https://raw.githubusercontent.com/NKwolv/agent-capmap/main/examples/demo/inventory.json
agentcapmap evaluate inventory.json --format markdown

# 2. List the MCP servers configured on your own machine (reads files only)
agentcapmap scan --user
```

On Windows PowerShell, use `curl.exe` instead of `curl`.

Step 1 should report 2 critical, 2 high and 1 medium finding. Step 2 lists your
servers with every secret redacted, without starting anything.

## Example output

```markdown
### 1. `CAP-001` | Critical

`Lethal trifecta: the agent can read private data, ingest attacker-controlled content,
and send data externally, so injected instructions can exfiltrate data.`

- **Agent:** `generic:fixtures/trifecta/mcp.json` at `fixtures/trifecta/mcp.json:3`
- **Tools:** `github/get_file_contents`, `github/list_issues`, `mailer/send_email`

| Server | Tool | Matched | Evidence |
|---|---|---|---|
| `github` | `get_file_contents` | `data_access=private_read` | `contents of a file from a private or public repository` |
| `github` | `list_issues` | `untrusted_input=true` | `including issues opened by anyone` |
| `mailer` | `send_email` | `egress=external` | `Send an email to any recipient address` |
```

(Abridged from the real output of step 1.)

## How it works

```
config files ──► discover ──► enumerate ──► classify ──► policy ──► report
                 (read only)  (initialize +  (LLM tags    (YAML      (JSON, Markdown,
                              tools/list)    6 axes)      rules)     SARIF, exit code)
```

1. **Discover** MCP configs: Claude Desktop, Cursor, VS Code (`.vscode/mcp.json`,
   `settings.json`), Claude Code (`.mcp.json`, and with `--user` your `~/.claude.json`
   user and per-project servers, including permission rules) and generic `mcp.json`.
   Secrets are redacted everywhere.
2. **Enumerate** (`--enumerate`): start each server and ask only for its tool list.
   Tools are never called.
3. **Classify** (`--classify`): an LLM tags each tool on six axes and flags tool
   descriptions that try to manipulate the model. Results are cached and validated
   against a JSON schema.
4. **Evaluate** (`--evaluate`): deterministic rules decide what is dangerous. The LLM
   never makes risk decisions.
5. **Report**: JSON, Markdown, or SARIF for GitHub code scanning; `--fail-on` sets the
   exit code.

### Full scan of your own setup (needs an API key)

```sh
export ANTHROPIC_API_KEY=<YOUR_API_KEY>          # PowerShell: $env:ANTHROPIC_API_KEY = "..."
agentcapmap scan --user --enumerate --classify --evaluate --format markdown -o report.md
```

`--enumerate` **runs the server commands in your configs**: only use it on configs you
trust. To try the whole pipeline on two known servers first, see
[`examples/live-trifecta`](https://github.com/NKwolv/agent-capmap/blob/main/examples/README.md).

## Capability taxonomy

| Axis | Values | Question |
|---|---|---|
| `data_access` | none, private_read, secrets_read | What non-public data can it read? |
| `untrusted_input` | false, true | Does it return content strangers can write? |
| `egress` | none, internal, external | Can it send data somewhere an outsider can read? |
| `mutation` | none, reversible_write, destructive | What can it change or delete? |
| `execution` | none, sandboxed, host | Does it run code, and where? |
| `privilege` | none, identity_admin | Does it manage users or permissions? |

Definitions and edge-case rulings: [docs/taxonomy.md](https://github.com/NKwolv/agent-capmap/blob/main/docs/taxonomy.md).

## Policy

The built-in [`policies/default.yaml`](https://github.com/NKwolv/agent-capmap/blob/main/policies/default.yaml) has nine rules:

| ID | Severity | Fires when an agent has |
|---|---|---|
| CAP-001 | critical | private data + untrusted input + external egress (lethal trifecta) |
| CAP-002 | critical | untrusted input + host code execution |
| CAP-003 | high | a destructive or host-execution tool with no approval gate |
| CAP-004 | high | secrets access + external egress |
| CAP-005 | high | a remote server with no authentication exposing capabilities |
| CAP-006 | high | tool definitions that changed since the baseline (rug pull) |
| CAP-007 | medium | a server not pinned to an exact version, or from an unvetted source |
| CAP-008 | high | a tool description that tries to instruct the model (tool poisoning) |
| CAP-009 | medium | the same tool name on two servers (shadowing) |

Write your own rules in the same format and pass `--policy my-policy.yaml`:

```yaml
- id: ACME-001
  severity: high
  when_server: {remote: true}
  when_agent_has_any:
    - privilege: identity_admin
  message: Remote servers must not manage identities.
```

Conditions: `when_agent_has_all`, `when_agent_has_any`, `when_server`,
`when_hash_changed`, `when_tool_name_shadowed`. The format is defined by
[`schemas/policy.schema.json`](https://github.com/NKwolv/agent-capmap/blob/main/schemas/policy.schema.json). Rug-pull detection:

```sh
agentcapmap baseline create inventory.json -o baseline.json   # once, after review
agentcapmap evaluate new-inventory.json --baseline baseline.json
```

To see what changed between two runs (for example `main` versus a pull request), save
each `evaluate` output and compare them:

```sh
agentcapmap diff before.json after.json --format markdown
agentcapmap diff before.json after.json --fail-on high --fail-on-new-capability
```

`diff` lists new and resolved findings and capabilities an agent gained or lost.
`--fail-on` looks at new findings only, so existing, already-accepted findings do not
fail the build; `--fail-on-new-capability` fails when an agent gains any capability.

## CI and GitHub code scanning

```sh
agentcapmap evaluate inventory.json --format sarif -o agentcapmap.sarif --fail-on high
```

Exit codes: `0` passed, `1` findings at or above `--fail-on`, `2` usage or input error.
The repository is also a GitHub Action: see [docs/github-action.md](https://github.com/NKwolv/agent-capmap/blob/main/docs/github-action.md)
and the [example workflow](https://github.com/NKwolv/agent-capmap/blob/main/docs/examples/agentcapmap.yml). By default the Action reads
configs only, which is safe for pull requests from forks.

## Accuracy

The classifier is evaluated against hand-labeled tools from real MCP servers (see
[`eval/`](https://github.com/NKwolv/agent-capmap/blob/main/eval/README.md)). **No accuracy numbers are published yet.** The 381 labels
(28 servers) were drafted by one AI model and then reviewed by another AI model at the
maintainer's request; no person has independently reviewed them yet. A model grading a
model can overstate accuracy, so numbers will be quoted here only after a person has
reviewed the labels and the run is frozen (`python eval/run_eval.py`). The same labels
also power classification without an API key for these known tools; see
[Limitations](#limitations).

## Limitations

- **Descriptions, not behavior.** Tools are classified by what their definitions say. A
  tool that lies about itself will be mis-tagged; agent-capmap never calls tools.
- **Classifier errors.** Tags can be wrong, especially for vague descriptions. Every
  tag carries a confidence and an evidence phrase; phrases not found in the definition
  are marked unverified.
- **Approval gates are mostly invisible.** Claude Code's permission settings are read
  (so its default "ask before each MCP call" counts as a gate), but most other clients do
  not record this, and CAP-003 treats "unknown" as "no gate" by default.
- **Enumeration runs code.** Starting servers executes the configured commands. A
  started server can also read its parent's environment on Linux.
- **Coverage.** Config formats evolve; Windsurf is not read yet. MCP servers that do not
  live in config files (Claude Code plugins, claude.ai connectors, IDE built-in servers,
  managed settings) are not seen. WebSocket servers are listed but not enumerated.
  Token scopes are not visible.
- **Reference labels are AI-reviewed.** For 381 tools from 28 well-known servers, the
  classifier answers from bundled labels (matched by exact definition hash) instead of
  calling a model, so no API key is needed. Those labels were reviewed by an AI model, not
  yet by a person, and a wrong label is returned with full confidence; any change to a
  tool's definition falls back to the cache or the LLM. Treat them as a convenience.
- **Not assessed is not safe.** Servers that could not be enumerated and tools that
  could not be classified are listed under `not_assessed`, never assumed safe.

## Safety

Secrets from configs are redacted in all output and logs. Enumeration sends only
`initialize` and `tools/list`, with a minimal environment for launched servers, no
credential-forwarding redirects, and size limits on everything a server returns.
Reports render untrusted text inertly. See [SECURITY.md](https://github.com/NKwolv/agent-capmap/blob/main/SECURITY.md) and the
[threat model](https://github.com/NKwolv/agent-capmap/blob/main/docs/threat-model.md).

## Documentation

- [Capability taxonomy](https://github.com/NKwolv/agent-capmap/blob/main/docs/taxonomy.md)
- [Threat model](https://github.com/NKwolv/agent-capmap/blob/main/docs/threat-model.md): attacks covered, OWASP / MITRE ATLAS mapping
- [GitHub Action](https://github.com/NKwolv/agent-capmap/blob/main/docs/github-action.md)
- [Comparison with Snyk Agent Scan (mcp-scan)](https://github.com/NKwolv/agent-capmap/blob/main/docs/vs-mcp-scan.md)
- [Examples](https://github.com/NKwolv/agent-capmap/blob/main/examples/README.md) · [Evaluation](https://github.com/NKwolv/agent-capmap/blob/main/eval/README.md) · [POC checklist](https://github.com/NKwolv/agent-capmap/blob/main/docs/poc-checklist.md)
- [Contributing](https://github.com/NKwolv/agent-capmap/blob/main/CONTRIBUTING.md) · [Security policy](https://github.com/NKwolv/agent-capmap/blob/main/SECURITY.md) · [Changelog](https://github.com/NKwolv/agent-capmap/blob/main/CHANGELOG.md)

## License

MIT. See [LICENSE](https://github.com/NKwolv/agent-capmap/blob/main/LICENSE).
