Metadata-Version: 2.4
Name: harpocrates
Version: 0.4.1
Summary: ML-powered secrets detection tool
Home-page: https://github.com/Skipa776/Harpocrates
Author: Joshua Do
Author-email: Joshua Do <joshalfonsodo@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/Skipa776/Harpocrates
Project-URL: Repository, https://github.com/Skipa776/Harpocrates
Project-URL: Issues, https://github.com/Skipa776/Harpocrates/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: typer>=0.9.0
Requires-Dist: rich>=13.0.0
Requires-Dist: onnxruntime>=1.17.0
Requires-Dist: numpy>=1.24.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: ruff>=0.6.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: build>=1.0.0; extra == "dev"
Provides-Extra: ml
Requires-Dist: xgboost<4,>=2.0.0; extra == "ml"
Requires-Dist: lightgbm<5,>=4.0.0; extra == "ml"
Requires-Dist: scikit-learn>=1.3.0; extra == "ml"
Requires-Dist: numpy>=1.24.0; extra == "ml"
Requires-Dist: pandas>=2.0.0; extra == "ml"
Requires-Dist: onnxmltools>=1.12.0; extra == "ml"
Requires-Dist: skl2onnx>=0.5.0; extra == "ml"
Provides-Extra: api
Requires-Dist: fastapi>=0.109.0; extra == "api"
Requires-Dist: uvicorn[standard]>=0.27.0; extra == "api"
Requires-Dist: pydantic-settings>=2.1.0; extra == "api"
Requires-Dist: python-multipart>=0.0.6; extra == "api"
Provides-Extra: mcp
Requires-Dist: mcp>=1.0.0; extra == "mcp"
Provides-Extra: llm
Provides-Extra: all
Requires-Dist: xgboost<4,>=2.0.0; extra == "all"
Requires-Dist: lightgbm<5,>=4.0.0; extra == "all"
Requires-Dist: scikit-learn>=1.3.0; extra == "all"
Requires-Dist: numpy>=1.24.0; extra == "all"
Requires-Dist: pandas>=2.0.0; extra == "all"
Requires-Dist: onnxmltools>=1.12.0; extra == "all"
Requires-Dist: skl2onnx>=0.5.0; extra == "all"
Requires-Dist: fastapi>=0.109.0; extra == "all"
Requires-Dist: uvicorn[standard]>=0.27.0; extra == "all"
Requires-Dist: pydantic-settings>=2.1.0; extra == "all"
Requires-Dist: python-multipart>=0.0.6; extra == "all"
Requires-Dist: mcp>=1.0.0; extra == "all"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

![Harpocrates](images/banner-a-classic-parchment.png)

<h1 align="center">Harpocrates</h1>

<p align="center">
  <strong>ML-powered secrets detection that catches what regex can't see</strong>
</p>

---

AI coding tools leak secrets at **2× the rate of human-written code** — 3.2% vs 1.6% of commits. 29 million secrets were exposed in 2025, up 34% year over year. One misconfigured environment variable cost a team **$87k in a single night**.

Regex scanners look for `AWS_ACCESS_KEY_ID` and `GITHUB_TOKEN`. They miss `client_secret`, `ENCRYPTION_KEY`, and `API_SECRET` — the names developers actually use. They miss secrets buried in comments. They miss env-var fallbacks. They miss the code your AI coding assistant just generated.

Harpocrates catches what slips through.

---

## Harpocrates vs TruffleHog

Benchmarked on a 5,000-line held-out evaluation set, cross-referenced against TruffleHog's output using 1,430 TruffleHog findings as ground truth.

| | TruffleHog | Harpocrates |
|---|---|---|
| Unique lines flagged | 606 | 1,300 |
| Lines only this tool caught | 449 | **1,143** |
| Lines both tools caught | 157 | 157 |
| Recall (held-out set) | — | **97.3%** |
| Precision (held-out set) | — | **87.1%** |
| Detection approach | Regex + live API verification | Regex + entropy + ML context |
| Scans comments | ❌ | ✅ |
| Catches ambiguous variable names | ❌ | ✅ |
| Catches env-var fallback leaks | ❌ | ✅ |
| Catches AI-scaffolded credential patterns | ❌ | ✅ |
| Classifies secret type (password, api\_token, jwt…) | ❌ | ✅ |
| Verifies credentials are live via API calls | ✅ | ❌ |

**Harpocrates caught 1,143 lines TruffleHog missed. TruffleHog caught 449 Harpocrates missed.**

TruffleHog's edge is live credential verification — it calls the AWS, GitHub, and Stripe APIs to confirm a key is still active. Harpocrates doesn't do that. If a key matches a known format and is still live, TruffleHog will catch it. Run both.

### What Harpocrates catches that TruffleHog can't

**1. Secrets under ambiguous variable names**

TruffleHog requires the variable name to match a known pattern. Harpocrates uses ML to evaluate the token, the variable name, and the surrounding context together.

| Code | TruffleHog | Harpocrates |
|------|-----------|-------------|
| `client_secret = "eyJhbGci..."` | ❌ | ✅ JWT (0.97) |
| `ENCRYPTION_KEY = "AKIAIOSFODNNabcd1234"` | ❌ | ✅ AWS key (0.90) |
| `API_SECRET = "AKIAJOE7MTPSO7EXAM1E"` | ❌ | ✅ AWS key (0.80) |
| `ACCESS_KEY = "eyJhbGciOiJSUzI1NiJ9..."` | ❌ | ✅ JWT (0.97) |
| `my_token = "xoxb-12345-67890-abcdef..."` | ❌ | ✅ Slack token (0.94) |

**2. Secrets buried in comments**

TruffleHog skips comment lines. Harpocrates scans all comment styles — `#`, `//`, `/* */`, `<!--`, `--` — and flags any high-entropy token it finds there.

```python
# old: DB_PASSWORD = "prod_hunter2_xxxSECRET"     ← TruffleHog: skipped. Harpocrates: caught.
// const apiKey = 'sk_live_NnyQ...';               ← TruffleHog: skipped. Harpocrates: caught.
TRACE_ID=$(git log) # AKIAMXWCUIXEKLN02TKZ         ← TruffleHog: sees git command. Harpocrates: sees the key in the comment.
```

**3. Environment variable fallback leaks**

A hardcoded fallback in `os.getenv()` is a secret that ships to production whenever the env var is unset. TruffleHog doesn't model this pattern. Harpocrates does.

```python
SECRET_KEY = os.getenv("SECRET_KEY", "hardcoded_prod_value_xxx")   # ← caught
api_key = config("API_KEY", default="sk_live_realkey123456789")     # ← caught
const key = process.env.KEY ?? "literal_secret_value";             # ← caught
```

**4. Dev/prod swap comments**

Rotating from a test key to a production key and leaving the prod key in a comment is a common pattern. The comment leaks the live credential.

```python
STRIPE_KEY = "sk_test_abc"  # PROD: sk_live_xyz_the_real_one    ← caught
```

**5. AI-scaffolded code**

AI coding assistants frequently hardcode credentials when scaffolding integrations. Harpocrates is trained on AI-generated code patterns specifically for this.

```python
# Generated by Copilot:
client = OpenAI(api_key="sk-proj-xLMN3kJqPv...")    # ← caught
stripe.api_key = "sk_live_9QnKsF4jQ8M..."           # ← caught
```

**6. Cross-language coverage**

The same ML model covers Python, JavaScript/TypeScript, Go, Java, Ruby, YAML, Dockerfile, Terraform, and Helm values files. TruffleHog's pattern library varies by detector; Harpocrates's entropy+ML layer is language-agnostic.

### Where TruffleHog has the edge

- **Live verification** — TruffleHog calls provider APIs to confirm a secret is active. Harpocrates never makes network calls.
- **Format-anchored patterns** — credentials that match a strict format (GitHub PAT prefix `ghp_`, Stripe's `sk_live_`) are caught with zero false positives by TruffleHog's regex. Harpocrates catches these too, but TruffleHog also confirms liveness.

**Recommended setup:** run Harpocrates as a pre-commit hook (catches the full range before anything reaches git) and TruffleHog in CI (confirms active credentials that made it through).

---

![Installation](images/divider-installation.png)

## Install

```bash
pip install harpocrates
harpocrates scan .
```

With ML verification (recommended — 87% precision, 97% recall on held-out real-world set):

```bash
pip install "harpocrates[ml]"
harpocrates scan . --ml
```

With REST API server:

```bash
pip install "harpocrates[api]"
harpocrates serve
```

Everything at once:

```bash
pip install "harpocrates[all]"
```

---

![Usage](images/divider-usage.png)

## Usage

```bash
# Scan a directory
harpocrates scan ./my_project

# Scan a single file
harpocrates scan config.env

# Output as JSON (pipe-friendly)
harpocrates scan ./my_project --json

# Enable ML verification to suppress false positives
harpocrates scan ./my_project --ml

# Only fail CI on high or critical findings
harpocrates scan ./my_project --fail-on high

# Ignore specific patterns
harpocrates scan ./my_project --ignore "*.test.js,fixtures/*"
```

### Pre-commit hook

```yaml
repos:
  - repo: https://github.com/Skipa776/Harpocrates
    rev: v0.2.0
    hooks:
      - id: harpocrates
```

Add to `.pre-commit-config.yaml`, then run `pre-commit install`. Harpocrates scans every staged file before each commit.

---

![Configuration](images/divider-configuration.png)

## Configuration

### `harpocrates scan` flags

| Flag | Default | Description |
|------|---------|-------------|
| `--ml` | off | Enable ML verification to reduce false positives |
| `--ml-threshold FLOAT` | `0.19` | ML confidence threshold `0.0–1.0`. Lower = more recall, higher = more precision |
| `--fail-on LEVEL` | `medium` | Severity that triggers exit code `1`: `critical` \| `high` \| `medium` \| `low` \| `info` \| `none` |
| `--json` | off | Output results as JSON instead of a table |
| `--show-secrets` | off | Print full token values instead of redacted previews |
| `--ignore TEXT` | — | Comma-separated glob patterns to skip (e.g. `"*.test.js,fixtures/*"`) |
| `--max-size INTEGER` | `10` | Maximum file size to scan, in MB |
| `--recursive / --no-recursive` | `--recursive` | Scan subdirectories recursively |

### Exit codes

| Code | Meaning |
|------|---------|
| `0` | No findings at or above `--fail-on` severity |
| `1` | One or more findings detected at or above `--fail-on` severity |
| `2` | Error (bad argument, unreadable file, etc.) |

### Other commands

```bash
harpocrates version          # Print version
harpocrates serve            # Start the REST API server (requires harpocrates[api])
harpocrates --help           # Full command list
```

---

![How it scans](images/divider-how-it-scans.png)

## How it scans

Harpocrates runs a three-phase pipeline on every line of every file:

1. **Regex** — deterministic patterns for known credential formats (AWS, GitHub, Stripe, private keys, and more). No ML required. High-confidence, zero false positives on well-formed keys.

2. **Entropy analysis** — Shannon entropy flags high-randomness tokens that don't match any known pattern. Catches credentials stored under ambiguous variable names (`my_key`, `token`, `secret`) that regex scanners miss entirely.

3. **ML verification** (opt-in via `--ml`) — a single-stage XGBoost classifier extracts 64 features from the token, its variable name, and the surrounding code context. It learns to distinguish `api_secret = "AKIA..."` (secret) from `commit_sha = "a1b2c..."` (Git SHA) without relying on the variable name alone. Inference runs via ONNX Runtime when available, with native XGBoost as fallback.

**Ships pre-trained. No user training required.**

The ML model is bundled with the package. `pip install "harpocrates[ml]"` is all you need.

---

## MCP Server

Harpocrates ships an MCP (Model Context Protocol) server that exposes `scan_text` and `scan_file` to compatible agents via stdio transport.

**Install:**
```bash
pip install "harpocrates[mcp]"   # requires Python 3.10+
```

**Run:**
```bash
harpocrates-mcp                  # listens on stdio, speaks JSON-RPC
```

**Tools exposed:**

| Tool | Arguments | Returns |
|------|-----------|---------|
| `scan_text` | `text: str`, `include_token: bool = False`, `include_contributions: bool = False` | List of finding dicts |
| `scan_file` | `path: str`, `include_token: bool = False`, `max_bytes: int\|None = None`, `include_contributions: bool = False` | List of finding dicts |

Tokens are redacted by default (`include_token=False`). Pass `include_contributions=True` to receive TreeSHAP explanations (requires `pip install harpocrates[ml]`). The MCP process has the same filesystem read scope as the user who launched it.

---

## Explainability

Harpocrates ships opt-in TreeSHAP explanations for ML-stage findings via `--explain`. The default scan path is unchanged — XGBoost is not imported unless `--explain` is passed.

**Install:**
```bash
pip install "harpocrates[ml]"
```

**CLI:**
```bash
# Emit JSON with per-feature SHAP contributions (implies --ml)
harpocrates scan ./my_project --explain

# Inspect the top features driving the first finding
harpocrates scan ./my_project --explain | jq '.findings[0].explanation.top_positive'
```

**Output shape:**
```json
{
  "findings": [
    {
      "finding": { "type": "ML_CANDIDATE", "severity": "high", "category": "api_token", ... },
      "explanation": {
        "finding_id": "a1b2c3d4e5f6a7b8",
        "category": "api_token",
        "base_log_odds": -1.4,
        "raw_log_odds": 2.8,
        "predicted_probability": 0.943,
        "top_positive": [
          { "name": "var_ngram_secret_score", "index": 12, "value": 0.873, "contribution": 1.85, "direction": "positive" },
          { "name": "token_entropy",          "index": 3,  "value": null,  "contribution": 1.10, "direction": "positive" }
        ],
        "top_negative": [ ... ],
        "contributions": [ ... ]
      }
    }
  ]
}
```

`explanation` is `null` for regex-tier findings (they have no model decision to explain). Token-derived feature values (`token_entropy`, `token_length`, etc.) are suppressed to `null` in the output to prevent token reconstruction.

**MCP:** pass `include_contributions=true` to `scan_text` or `scan_file` to receive explanations inline.

---

## Version history

### v0.4.0 — Redesigned 64-feature vector, v0.4 retrain, opt-in TreeSHAP explainability

**Feature engineering (Phase 7.0 / 7.0.11):**

- **Dropped 8 shortcut / collinear features** that caused the model to memorise narrow synthetic distributions rather than generalise. Dropped: `var_contains_secret` (25% importance but a direct mirror of the heuristic layer — label leak), `is_known_hash_length` (inverted signal on real credentials), `cryptographic_score`, `normalized_entropy` (collinear with `token_entropy` + char counts), `line_position_ratio` (train/serve skew from synthetic estimation), `hex_context_git_keywords`, `cross_line_entropy`, `contains_example_keyword` (collinear with their sibling features). Net feature count: 65 → 64.
- **Added 7 value-shape features** that distinguish credentials from file paths, enum constants, host configs, and template placeholders — the four false-positive classes from real-world scans: `value_starts_with_slash`, `value_contains_path_separator`, `value_ends_with_known_ext`, `value_is_lowercase_word`, `value_is_dotted_quad_or_host_literal`, `value_is_template_syntax`, `is_hex_with_no_alpha_mix`.
- **Tightened XGBoost regularisation** (`max_depth` 6→5, `n_estimators` 500→300, L1/L2 increased) to break reliance on shortcut features.

**v0.4 retrain (Phase 7):**

- Retrained on ~40k samples (v4 corpus) with the redesigned 64-feature vector.
- New negative-class generators covering the four v0.3.0 FP classes: file-path values, enum constants (lowercase words), host/port configs, template placeholders (`${...}`, `{{...}}`, `__X__`), file-extension values (`.pem`, `.jks`, `.p12`).
- New positive-class generators: APIM-style compound var names (`APIM_CLIENT_KEY`, `APIM_SECRET_KEY`), multiline structures (PEM blocks, YAML pipe-block secrets, k8s `Secret` manifests, `.env` multikey blocks, JSON arrays of keys, Terraform `for_each` secret maps) encoded via `context_before`/`context_after`.
- 300-sample hand-curated positive holdout fixture (`tests/fixtures/positive_holdout_v4.jsonl`); golden OOD recall **97.33%** (gate: ≥97%).

**Opt-in explainability (Phase 8):**

- `--explain` flag on `harpocrates scan` emits structured JSON with per-feature TreeSHAP contributions. Implies `--ml`. Default scan path unchanged — xgboost is never imported without `--explain`.
- `include_contributions=True` on MCP `scan_text`/`scan_file` tools adds an `explanation` key to each finding dict.
- All float outputs rounded to 3 decimal places. Token-derived feature values (`token_entropy`, `token_length`, `char_class_count`, `digit_ratio`, `special_char_ratio`) suppressed to `null` in JSON output to prevent token reconstruction.
- Booster cached per process (first `--explain` call ~30ms; subsequent calls ~0.2ms each).
- Hot-path invariant enforced by `tests/test_hot_path_no_xai.py` (subprocess isolation + AST static analysis).

**Severity calibration (v0.3.2, folded into this release train):**

- `_entropy_severity()` stub replaced with a classification-aware helper — entropy/ML findings now surface at MEDIUM or HIGH instead of always INFO.
- Suffix-style var-name lexicon added (`*_KEY`, `*_SECRET_KEY`, `*_CLIENT_KEY`, etc.) so `APIM_CLIENT_KEY`, `APIM_SECRET_KEY` classify as `api_token` at confidence 0.80 → MEDIUM.
- `OPENAI_API_KEY_LEGACY` regex added to `HIGH_SIGNATURES` — catches legacy/short OpenAI keys with hyphens (`sk-RQMJj8ELDjv7TRc-...`) missed by the strict 48-char CRITICAL pattern.

---

### v0.3.0 — ML model retrained on 62k samples, comment scanning, violation classification

**New ML capabilities (what the model can now detect that it couldn't before):**

- **Commented-out secrets** — all comment styles now scanned: `#` (Python/YAML/Ruby), `//` (JS/TS/Go/Java), `/* */` (C/CSS/Java), `<!--` (HTML), `--` (SQL/Lua). A credential commented out of active code is still in your git history forever.
- **AI-scaffolded credentials** — trained on LLM-generated code patterns where AI assistants hardcode credentials during integration scaffolding (OpenAI, Stripe, Twilio, Terraform, Helm, Dockerfile).
- **Runtime generation negatives** — the model now correctly ignores `secrets.token_hex(32)`, `crypto.randomBytes(32)`, `uuid.uuid4().hex`, `SecureRandom.getInstanceStrong()`. These are runtime generation calls, not stored secrets. v0.2 would flag them.
- **Dev/prod swap patterns** — `KEY = "sk_test_x"  # PROD: sk_live_y` — the trailing comment leaks the production key.
- **Duplicate-token lines** — `password = "password123"`, `token = "token-prod-abc..."`. The ML model now handles lines where the variable name appears as a substring of the value, which confused earlier models.
- **Env-var fallback leaks** — expanded coverage across Django (`config(..., default=)`), Pydantic Settings, Spring Boot `@Value`, Rails `credentials.dig`, Go, Node.js (`??`), TypeScript/Zod, Helm `| default`.
- **Violation classification** — every finding now includes a `category` field (`password`, `api_token`, `connection_string`, `jwt`, `private_key`, `crypto_key`, `oauth_secret`, `session_token`, `webhook_url`, `generic_secret`) and a `category_reason` string explaining which heuristic fired. This tells you what to rotate, not just that something was found.

**Model metrics (v5 corpus, 62k training samples):**

| Set | Recall | Precision | F1 | AUC-ROC |
|-----|--------|-----------|-----|---------|
| Synthetic test (internal) | 99.15% | 94.61% | 0.968 | 0.9957 |
| Held-out OOD (TruffleHog-validated) | **97.32%** | **87.14%** | 0.920 | 0.9823 |

Threshold low (recall gate): `0.15` — anything above this is flagged for review.
Threshold high (precision gate): `0.85` — anything above this is reported as a confirmed secret.

---

### v0.2.0 — Three-phase pipeline, MCP server, pre-commit hook

**ML capabilities added:**

- **XGBoost + ONNX Runtime** — single-stage classifier with 64 features. Replaces the v0.1 heuristic confidence scores with a calibrated probability (Platt-scaled).
- **64-feature context model** — captures token entropy, variable name semantics (`var_ngram_secret_score`), file type risk (`file_is_config`, `file_extension_risk`), surrounding context (`context_has_function_def`), and value structure (`is_hex_with_no_alpha_mix`, `value_starts_with_slash`, `value_is_dotted_quad_or_host_literal`, `value_is_template_syntax`).
- **Dual-threshold routing** — findings are routed to `SAFE` (below threshold\_low), `REVIEW` (between thresholds), or `SECRET` (above threshold\_high). The review zone is ~5% of lines on average.
- **Entropy candidates surfaced** — high-entropy tokens under any variable name now reach the ML stage, not just those matching known patterns.
- **MCP server** — `harpocrates-mcp` exposes `scan_text` and `scan_file` as JSON-RPC tools for agent-to-agent scanning (Claude, Cursor, Codeium, etc.).
- **TokenMatch infrastructure** — internal span tracking for exact token position within a line, required for the v0.3 feature-set retrain.

**Regex signatures added in v0.2:**

| Credential | Pattern |
|-----------|---------|
| OpenAI API key | `sk-...` / `sk-proj-...` |
| Anthropic Claude key | `sk-ant-api03-...` |
| GCP API key | `AIza...` |
| NPM token | `npm_...` |
| PyPI token | `pypi-...` |
| HashiCorp Vault token | `hvs....` |

---

### v0.1.0 — Initial release ("the wedge")

**What shipped:**

- Three-phase detection pipeline: Regex → Shannon Entropy → ML confidence scoring.
- 10 regex signatures: AWS Access Key ID, GitHub PAT, Slack token, Stripe key, private key PEM header, Slack webhook, Discord webhook, SendGrid key, Twilio SID, Databricks token.
- Pre-commit hook — scans every staged file before commit. Handles binary files and `UnicodeDecodeError` without crashing.
- CLI: `harpocrates scan`, `--json`, `--fail-on`, `--ml-threshold`, `--show-secrets`, `--ignore`.
- Packaged as `py3-none-any.whl` — no native extensions, installs on any platform without compilation.

---

![Contributing](images/divider-contributing.png)

## Contributing

**False negatives are the highest-priority reports.** If Harpocrates missed a real secret, [open an issue with the `false-negative` label](https://github.com/Skipa776/Harpocrates/issues/new?labels=false-negative) and include the variable name pattern and secret type. This is the most valuable feedback you can give.

For bugs, feature requests, and false positives, open an issue at [github.com/Skipa776/Harpocrates/issues](https://github.com/Skipa776/Harpocrates/issues).

---

![License](images/divider-license.png)

## License

MIT — see [LICENSE](LICENSE).
