Metadata-Version: 2.4
Name: api-key-scanner-mcp
Version: 0.2.0
Summary: Verify LLM API gateway authenticity. Local MCP server, your API key stays on your machine.
Project-URL: Homepage, https://github.com/zhonghp/api-key-scanner
Project-URL: Documentation, https://github.com/zhonghp/api-key-scanner/tree/main/docs
Project-URL: Issues, https://github.com/zhonghp/api-key-scanner/issues
Author: api-key-scanner contributors
License:                                  Apache License
                                   Version 2.0, January 2004
                                http://www.apache.org/licenses/
        
           Licensed under the Apache License, Version 2.0 (the "License");
           you may not use this file except in compliance with the License.
           You may obtain a copy of the License at
        
               http://www.apache.org/licenses/LICENSE-2.0
        
           Unless required by applicable law or agreed to in writing, software
           distributed under the License is distributed on an "AS IS" BASIS,
           WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
           See the License for the specific language governing permissions and
           limitations under the License.
        
           Copyright 2026 api-key-scanner contributors
        
           Licensed under the Apache License, Version 2.0 (the "License");
           you may not use this file except in compliance with the License.
           You may obtain a copy of the License at
        
               http://www.apache.org/licenses/LICENSE-2.0
        
           Unless required by applicable law or agreed to in writing, software
           distributed under the License is distributed on an "AS IS" BASIS,
           WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
           See the License for the specific language governing permissions and
           limitations under the License.
License-File: LICENSE
Keywords: anthropic,api-verification,claude-code,llm,mcp,openai
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Requires-Dist: httpx>=0.27.0
Requires-Dist: mcp>=1.2.0
Requires-Dist: platformdirs>=4.0.0
Requires-Dist: pydantic>=2.6.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: sigstore>=3.0.0
Requires-Dist: tiktoken>=0.7.0
Provides-Extra: dev
Requires-Dist: mypy>=1.10.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: respx>=0.21.0; extra == 'dev'
Requires-Dist: ruff>=0.5.0; extra == 'dev'
Requires-Dist: vcrpy>=6.0.0; extra == 'dev'
Description-Content-Type: text/markdown

# api-key-scanner

Verify whether an LLM API gateway (or 中转站 / third-party proxy) is
actually serving the model it claims — without ever handing your API
key to anyone else.

**[简体中文](./README.zh-CN.md)**

## How it works

You point the tool at an endpoint, tell it what model the gateway
claims to serve, and name the environment variable where your gateway
API key lives. It runs a small probe set against the endpoint, compares
the responses to publicly-signed reference fingerprints of real vendor
models, and returns a trust score.

Everything runs on your machine. Your API key is read from a local env
var (never accepted in chat, never sent anywhere except the gateway you
named). Reference fingerprints are downloaded from our GitHub Releases
and Sigstore-verified before use — if the signature doesn't match the
identity of the GitHub Actions workflow that produced them, the tool
refuses the data and returns `inconclusive`.

The tool ships two pieces: an MCP server (on PyPI) and a skill that
teaches the agent when and how to call the server's `verify_gateway`
tool. Both pieces are platform-neutral; installation just differs per
host.

## Prerequisites

The MCP server runs through [`uvx`](https://docs.astral.sh/uv/), which
fetches and isolates the Python package on every launch. If you don't
already have `uv`, install it once:

```bash
# macOS
brew install uv

# macOS / Linux (no Homebrew)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows (PowerShell)
irm https://astral.sh/uv/install.ps1 | iex
```

After install, `uvx --version` should print a version. Restart your
terminal (or your MCP client) if it doesn't show up yet.

## Installation

### Claude Code

In Claude Code:

```
/plugin marketplace add zhonghp/api-key-scanner
/plugin install api-key-scanner@zhonghp-api-key-scanner
```

That's everything — the plugin bundles the MCP server config and the
skill. The server auto-downloads signed reference fingerprints on first
use; no manual bootstrap.

### OpenClaw

Two steps.

**1. Add the MCP server to your OpenClaw config:**

```jsonc
{
  "mcp": {
    "servers": {
      "api-key-scanner": {
        "command": "uvx",
        "args": [
          "--refresh-package", "api-key-scanner-mcp",
          "api-key-scanner-mcp@latest"
        ]
      }
    }
  }
}
```

**2. Install the skill so OpenClaw knows when to invoke it:**

```bash
mkdir -p ~/.openclaw/skills/api-verify
curl -fsSL \
  https://raw.githubusercontent.com/zhonghp/api-key-scanner/main/skills/api-verify/SKILL.md \
  -o ~/.openclaw/skills/api-verify/SKILL.md
```

Restart OpenClaw. That's it.

### Other MCP clients (Cursor, Continue, Zed, Claude Desktop, …)

Any MCP client works — the server is just a standard stdio JSON-RPC
server. Point your client's MCP config at `uvx --refresh-package
api-key-scanner-mcp api-key-scanner-mcp@latest` and you're done. Skills
are Claude Code / OpenClaw-specific; on other clients the user just
mentions the `verify_gateway` tool directly.

## Supplying your gateway key

The server reads your gateway's API key from the local environment.
Put it in `~/.api-key-scanner/.env`:

```bash
mkdir -p ~/.api-key-scanner
cat > ~/.api-key-scanner/.env <<'EOF'
MY_GATEWAY_KEY=sk-your-gateway-key
EOF
```

The server auto-loads this file at startup. A shell `export` works too,
but only if you set it *before* launching the MCP client — most clients
(Claude Code, OpenClaw, etc.) snapshot env at launch.

## Using it

Ask in natural language:

> 帮我验证下 `https://api.example.com/v1` 提供的 gpt-4o 模型是不是
> 真的。我的 key 放在 `MY_GATEWAY_KEY` 环境变量里。

(Phrase the question as "is the model at this URL genuine?", not "is
this URL gpt-4o?" — the former makes the authenticity question clear.)

You get a verdict with:

- `trust_score` — 0.0 to 1.0
- `verdict` — `ok` / `suspicious` / `likely_substituted` / `inconclusive`
- `confidence` — `low` / `medium` / `high`
- Detector breakdown (`d1_banner_match`, `d2_met`, `d4_metadata`)

Cutoffs:

- `>= 0.90` → responses are consistent with the claimed model
- `0.70 – 0.90` → drift worth a deeper probe
- `< 0.70` → the model behind the endpoint likely isn't what's claimed
- `inconclusive` → the verdict explains which step failed

Budget: `cheap` (~18 calls, default — smoke test) or `standard`
(~258 calls, the full MET paper protocol). Higher = higher confidence,
more calls on your gateway. Start with `cheap` to sanity-check the
endpoint; escalate to `standard` when a suspicious result warrants
the extra statistical power.

## Which models can it verify?

Only models for which we've published a signed reference fingerprint.
The current coverage — with canonical IDs, vendor model names, and the
source endpoint each fingerprint was collected from — is tracked in
**[SUPPORTED_MODELS.md](./SUPPORTED_MODELS.md)**. If the model you care about isn't
listed there, `verify_gateway` will return `inconclusive` — the tool
doesn't guess.

For the live list at runtime, just ask in chat:

> What models does api-key-scanner currently support?

The agent calls `list_supported_models`, which reads the latest
release's `MANIFEST.json` directly — this will always be accurate even
if `SUPPORTED_MODELS.md` hasn't been updated yet.

## What it catches

- **Cross-family substitution** — endpoint claims gpt-4o but serves
  Llama / Qwen / Claude / Gemini.
- **Cached replay** — endpoint returns canned vendor-style answers
  instead of a real model call.
- **System-prompt tampering** — endpoint silently injects a hidden
  system prompt that changes behavior.

## What it doesn't catch (yet)

- **Same-family downgrade** — Opus → Sonnet → Haiku,
  gpt-4o → gpt-4o-mini. We sometimes flag `suspicious` but don't
  commit to a verdict.
- **Quantization** — same model at a lower precision. Academic
  results show black-box detection is ≈ random.
- **Adaptive routing** — the gateway plays real-model to "identify
  yourself" probes and cheap-model to everything else. Needs an
  out-of-band trust anchor to solve.

Treat any `likely_substituted` as a signal to dig deeper, not as proof
of fraud. Trust scores are statistical, not legal.

## Updates

| Something updates | What you need to do |
|---|---|
| New weekly fingerprint snapshot | Nothing — the server checks GitHub on each startup and auto-upgrades. |
| New MCP server version | Restart your MCP client; `uvx` pulls `@latest`. |
| Plugin manifest or skill changes | Claude Code: `/plugin marketplace update zhonghp-api-key-scanner`, then restart. OpenClaw: re-copy `SKILL.md` (see install). |
| Cache problems — old version won't let go | `uv cache clean api-key-scanner-mcp`, then restart. |

## Privacy

- **Your API key** is read locally from your `.env` or environment.
  Never logged, never sent anywhere except the target gateway you
  specified. Key substrings are scrubbed from any error that surfaces
  in verdicts.
- **Gateway responses** are analyzed in-process. Nothing is uploaded.
- **Outbound network from this tool** is limited to (1) the gateway
  URL you provided and (2) `github.com/zhonghp/api-key-scanner`
  releases for the signed reference data.
- **No backend, by design.** There's nowhere for us to log your
  traffic.

## Reference-data integrity

Fingerprint snapshots are signed with [Sigstore](https://www.sigstore.dev/)
keyless OIDC. The signing identity is bound to our weekly GitHub Actions
workflow. The server refuses to load a release whose signature identity
doesn't match — it degrades to `inconclusive` rather than return a
verdict based on unverified reference data.

## Air-gapped use

Download a fingerprint release on another machine, copy it over, then
point the server at the local directory:

```bash
export APIGUARD_FINGERPRINT_DIR=/path/to/fingerprint-YYYY-MM-DD
```

(Set this in `~/.api-key-scanner/.env` so the MCP client's subprocess
picks it up.) The Sigstore check is still enforced; you'll need the
`MANIFEST.json.sigstore.json` in the directory.

## Acknowledgments

The detection methodology in this project builds directly on two
open-source research efforts, both MIT-licensed. Enormous thanks to
their authors for making everything — code, weights, datasets —
publicly available.

- **[LLMmap](https://github.com/pasquini-dario/LLMmap)** — Dario
  Pasquini et al., *"LLMmap: Fingerprinting for Large Language
  Models"* (USENIX Security 2025, [arXiv:2407.15847](https://arxiv.org/abs/2407.15847)).
  Our D1 detector uses LLMmap's default 8-query strategy (from
  `confs/queries/default.json`). The current D1 implementation is a
  lightweight banner-match approximation — the full trained-classifier
  version is planned as a future upgrade.

- **[Model Equality Testing](https://github.com/i-gao/model-equality-testing)**
  — Irena Gao et al., *"Model Equality Testing: Which Model Is This
  API Serving?"* ([arXiv:2410.20247](https://arxiv.org/abs/2410.20247)).
  Our D2 detector vendors this project's MMD² Hamming kernel and
  permutation p-value procedure. The prompt style (Wikipedia
  continuation tasks) and the N=250 sample protocol both come from
  the paper's main experimental configuration.

## License

- **Code**: [Apache-2.0](LICENSE)
- **Fingerprint data** (GitHub Release assets): CC-BY-4.0
