# IntentProbe

> A research preview. A local scanner for MCP servers, tools, and skills that reads a frozen model's activations, not just the text. It generalizes to attack sources and wording it never trained on better than a same-data text classifier.

IntentProbe is a local CLI scanner, GitHub Action, and runtime hook for AI agent tools, MCP servers, and skills. It runs a tool description or prompt through a frozen Qwen2.5-0.5B, reads mean-pooled mid-layer activations (layers 13-15), and scores them with a small (~22 KB) logistic probe. The activation probe is the primary signal for allow/warn; the block tier additionally requires static-keyword corroboration.

## Key facts (all reproducible from research/)

- Real human attacks (HackAPrompt, n=3,866 uniform-random, a held-out source neither detector trained on): probe recall 90.3% at 5% clean-FPR and 88.3% at 1%, vs TF-IDF on the same training data 52.8% and 30.3%. Positive-only set, so this is recall at a matched FPR, not AUROC.
- Curated cross-source generalization (4 PI datasets, leave-one-source-out, nested CV): probe mean AUROC 0.984 vs TF-IDF 0.914; hardest held-out source deepset 0.941 vs 0.732 (+0.209, 95% CI [0.168, 0.25]). Shipped fixed config reaches mean 0.980.
- Tool poisoning is PARTIAL and on SYNTHETIC attacks (no real-human tool-poisoning corpus exists yet): MCPTox held-out 0.738 vs 0.545 (significant); minimal-pairs at chance for both.
- Within-distribution on matched-vocabulary minimal pairs the probe TIES TF-IDF (~0.79 vs ~0.82); text is not blind there. The edge is cross-source / novel-vocabulary, not same-vocab.
- NOT the first probe-based detector: PIShield, TaskTracker, RouteGuard, MindGuard, and frontier-lab production probes predate or parallel it. The only-one-we-found niche is the deployment shape (installable, pre-install, scans the tool description, on activations), not the technique.
- Runs 100% locally on CPU, nothing uploaded. ~22 KB probe head (a train/store advantage; inference needs the frozen 0.5B host, so it is heavier than a standalone text classifier). Apache-2.0.
- CLI: `intentprobe scan`, `scan-config auto`, `scan-path`, `batch`, `runtime`. GitHub Action: `mcpware/IntentProbe@main`. Runtime hook for Claude Code PreToolUse.
- Research preview, a local registration-time review signal, NOT a hard security boundary.

## Install

```
pip install intentprobe
intentprobe scan-config auto --format summary
```

## Links

- Repository: https://github.com/mcpware/IntentProbe
- Research paper (preliminary, GPT-2): https://doi.org/10.5281/zenodo.19990741
- Research results: research/_results_published/
- Competitive landscape: https://github.com/mcpware/IntentProbe/blob/main/docs/COMPETITIVE_LANDSCAPE.md
- License: Apache-2.0
