Metadata-Version: 2.4
Name: blastcontain-drill
Version: 0.2.0
Summary: Adversarial red-team scanner for AI agents — versioned attack corpus, cage action-ground-truth, signed DrillReport
Project-URL: Homepage, https://github.com/gdeudney/blastcontain-oss/tree/main/drill
Project-URL: Repository, https://github.com/gdeudney/blastcontain-oss
Project-URL: Issues, https://github.com/gdeudney/blastcontain-oss/issues
Project-URL: Documentation, https://github.com/gdeudney/blastcontain-oss/blob/main/drill/docs/spec.md
Project-URL: Changelog, https://github.com/gdeudney/blastcontain-oss/blob/main/drill/CHANGELOG.md
Author: BlastContain Contributors
License: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: agent-security,ai-safety,jailbreak,mcp,mitre-atlas,owasp-agentic,prompt-injection,red-team
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.11
Requires-Dist: blastcontain-core<1.0,>=0.2
Requires-Dist: click>=8.1
Requires-Dist: httpx>=0.27
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest-timeout>=2.3; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: full
Requires-Dist: deepeval>=1.0; extra == 'full'
Provides-Extra: judge
Requires-Dist: deepeval>=1.0; extra == 'judge'
Description-Content-Type: text/markdown

# blastcontain-drill

Adversarial red-team scanner for AI agents. Drill runs a **versioned attack
corpus** against an agent **inside a cage** and scores the result on two planes —
what the model *said* (content) and what the agent *did* (action). It produces a
signed, MITRE-ATLAS-tagged **DrillReport** in the same Audit-Packet format as
[`blastcontain-verify`](https://github.com/blastcontain/verify).

> The cage trilogy: **Verify** proves the cage is built right · **Drill** attacks
> the agent inside it · **Guard** adds the runtime locks.

```
pip install blastcontain-drill
blastcontain-drill --agent-id my-agent --cage inprocess \
  --target-base-url http://localhost:1234/v1 --target-model qwen/qwen3.6-27b \
  --report drill.md --output drill.json
```

## What makes Drill different

Existing tools score the **model's output** (jailbroken text, PII in a response).
Drill, running the agent in a cage, also scores the **action**: did the canary
*actually* leave? did a forbidden tool *fire*? did the agent *attempt* an egress
the cage blocked? *Content scoring says "the model said something bad." Drill says
"the agent did something bad."*

| Plane | Asks | How |
|---|---|---|
| **Content** | did the model *say* something bad? | LLM-as-judge + a guardrail classifier (Qwen3Guard or IBM Granite Guardian) |
| **Action** ★ | did the agent *do* something bad? | cage ground truth — canary exfil · forbidden-tool fire · egress attempt · tool-call log |

A scenario returns **HELD** or **BYPASS** with a detection latency and the control
that blocked it. An action-plane bypass is **CRITICAL** and blocks prod promotion.

## The attack corpus — three layers, versioned

| Layer | Source | Catches |
|---|---|---|
| **Replay** | built-in seeds · JailbreakBench · system-card checks · multi-turn vectors · AI-Infra-Guard curated sets | *known* attacks — a regression suite |
| **Operators** | arXiv techniques as transforms (PyRIT/DeepTeam encoders, framing, obfuscation, homoglyph…) | known *methods* on fresh seeds |
| **Generative** | an abliterated attacker model in a PAIR/TAP loop | *novel* jailbreaks |

Every run is pinned to a corpus version (e.g. `v2026.06.1`) and recorded in the
DrillReport, so reports are reproducible and regression-comparable.

**Replay sources (opt-in, mix per run):**

- Built-in seed corpus — always on, version-pinned (`BUILTIN_CORPUS_VERSION`), ATLAS-tagged.
- `--jbb` — JailbreakBench (100 harmful + 100 benign over-refusal probes, dataset revision pinned).
- `--systemcard` — cyber-misuse / dual-use, identity & leaked-info honesty, ART indirect-injection probes.
- `--multiturn` — long-context reference tracking, decomposition/recompose, and real multi-turn crescendo (a conversation carried across turns, not a single prompt).
- `--enable-aig` — AI-Infra-Guard as a live attack-source plugin (if the service is up).

**Operators (`--operators`)** applies model-free string transforms (base64, ROT13,
Caesar, Atbash, Morse, binary, url-encode, reverse, leetspeak, many-shot, persona,
payload-split, prefix-injection, multilingual, char-space, zero-width, homoglyph)
to every seed, expanding each into fresh variants while preserving its
category/goal. No model required — fast and reproducible.

**Generative (`--generative`)** runs an abliterated/no-refusal attacker model
against the caged target in a PAIR/TAP loop (`--attacker-model`). Discovered
jailbreaks are written to a separate, sensitive corpus (`--generative-corpus`),
never into the signed report (which carries only an excerpt).

## The cage

The cage is an interface with two backends:

- **`inprocess`** — host-side cage that drives a *real* model (any OpenAI-compatible
  endpoint, e.g. LM Studio) through an agent loop with a tool allowlist, an egress
  allowlist, a planted canary, and a tool-call log. Used for live runs and fast tests.
- **`podman`** — a real container with deny-all egress (`--network none`), proving
  containment ground truth. Runs a non-LLM stub agent in CI.

Both produce the same observations, so the **action probes** (canary / egress /
forbidden-tool) score identically against either.

## Taxonomy

Every finding is tagged with **MITRE ATLAS** (primary; verified against
atlas.mitre.org), plus the MIT AI Risk domain and OWASP Agentic `T#`. The agent
techniques `AML.T0086` (Exfiltration via AI Agent Tool Invocation) and `AML.T0110`
(AI Agent Tool Poisoning) cover the action plane. The maps live in
[`blastcontain-core`](https://github.com/blastcontain/core).

## CLI flags (most common)

```
--agent-id              Target agent identifier (required)
--env                   dev | uat | staging | prod
--cage                  inprocess | podman   (default: inprocess)
--target-base-url       OpenAI-compatible endpoint for the in-cage agent
--target-model          model id to drive as the agent (e.g. qwen/qwen3.6-27b)
--judge-base-url        endpoint for the LLM judge (defaults to --target-base-url)
--judge-model           model id for the content-plane judge
--judge-kind            llm | geval               (default: llm; geval needs [judge] extra)
--guard-model           guardrail classifier id (auto-selects Qwen3Guard / Granite Guardian / WildGuard)
--agent-url             attack an already-running agent over HTTP (black-box mode)
--corpus                corpus version to pin (default: built-in latest)
--scenarios             comma-separated categories (default: all)
--charter               local charter.yaml — its permitted_tools define "forbidden"

# Corpus source toggles (see "The attack corpus" above)
--jbb                   include JailbreakBench (100 harmful + 100 benign)
--systemcard            include system-card checks (cyber-misuse, honesty, ART indirect)
--multiturn             include multi-turn vectors (long-context, decomposition, crescendo)
--enable-aig            include AI-Infra-Guard as a live attack source
--operators             expand every seed with model-free technique transforms
--generative            run the generative attacker loop
--attacker-model        attacker model id (abliterated/heretic; required with --generative)
--generative-corpus     write discovered jailbreaks (SENSITIVE; separate from --output)

--output PATH           write the signed DrillReport JSON
--report PATH           write the Markdown report
```

## Verifying a DrillReport

```python
import json
from blastcontain_core.signing import verify_packet

packet = json.load(open("drill.json"))
assert verify_packet(packet)
```

Ed25519 packets carry their public key inline. Set `BLASTCONTAIN_SIGNING_KEY_PATH`
to an Ed25519 PEM for production attestation; otherwise an advisory HMAC is used.

## Container

```
# from the blastcontain-oss/ root (dev build — installs core from local source)
podman build -t blastcontain-drill:0.1.0 -f drill/Containerfile .

# black-box: attack a running agent
podman run --rm -v "$PWD/reports:/reports:rw" blastcontain-drill:0.1.0 \
  --agent-id my-agent --agent-url http://agent:8080 \
  --report /reports/drill.md --output /reports/drill.json
```

Drill drives a model, so — unlike Verify — it is **not** run with `--network none`.
To point the inprocess cage at a model server on the host, see the Containerfile header.

## Safety

Drill emits live adversarial payloads. Run the corpus and any attacker model in
the cage, treat generated jailbreaks as secrets, and check dataset licenses before
redistribution. See [SECURITY.md](SECURITY.md).

## License

[Apache 2.0](LICENSE). See [NOTICE](NOTICE).

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md). All contributions require a DCO sign-off (`git commit -s`).
