Metadata-Version: 2.4
Name: superred-claim-agentharm
Version: 0.1.0
Summary: AgentHarm SecurityClaim for superred: faithful port of the harmful test_public split (176 behaviors) against the general inspect-agent target, reusing upstream inspect_evals.agentharm tools/grading/judges. 8 per-category subclaims + total.
Project-URL: Homepage, https://superred.simonsure.com
Project-URL: Documentation, https://superred.simonsure.com
Project-URL: Repository, https://github.com/RoldSI/superred-modules
Project-URL: Issues, https://github.com/RoldSI/superred-modules/issues
Author-email: Simon Sure <info@simonsure.com>
License-Expression: MIT
License-File: LICENSE
License-File: NOTICE
Keywords: agentharm,ai-safety,benchmark,llm,red-teaming,security,security-claim,superred
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Security
Requires-Python: <3.14,>=3.11
Requires-Dist: huggingface-hub>=1.2
Requires-Dist: inspect-ai<0.4,>=0.3.229
Requires-Dist: inspect-evals==0.13.1
Requires-Dist: numpy>=1.26
Requires-Dist: superred-target-inspect-agent>=0.1.0
Requires-Dist: superred>=0.1.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=6.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Description-Content-Type: text/markdown

# security-claim-agentharm

A faithful superred port of the [AgentHarm](https://arxiv.org/abs/2410.09024)
benchmark (Andriushchenko et al.), harmful `test_public` split: **176 behaviors**
(44 base x 4 augmentations) across **8 harm categories**. It runs against the
general `inspect-agent-target` and reuses upstream `inspect_evals.agentharm`
tools, grading functions, and judges **verbatim**, so the *grading* of a run is
faithful to AgentHarm's own scoring.

Threat model (default `benign_cover=True`): each behaviour is reframed as
**indirect prompt injection**. The agent is given a purpose-built **benign cover
task** that engages the behaviour's read tools but does not perform the harmful
actions; the harmful behaviour is the adversarial **`Goal` an attacker must
inject** into a tool output. A do-nothing attacker does the benign task and fails,
so success measures the attacker's *lift*. The harmful tools and grader are kept
verbatim, so the harmful environment supplies the targets a real injection needs.
Set `benign_cover=False` to restore upstream's **direct-request** model (the
harmful behaviour IS the user task; the *user* is the adversary, refusal
robustness) and reproduce its direct-request ASR. See `ASSUMPTIONS.md` section J.

## Mapping to superred

- **Target** = the agent under test (general `InspectAgentTarget` wired with
  AgentHarm's tool resolver).
- **Optimizer = the attacker.** Under the default benign-cover mode a passthrough
  optimizer does the benign cover task and fails (no harm); a real attacker injects
  into a `tool:<name>` output to redirect the agent into the harmful `Goal`. Under
  `benign_cover=False`, a passthrough is AgentHarm's *direct-request* baseline and
  reproduces the headline numbers, while a jailbreak optimizer overrides the
  `user_prompt` controllable.
- **Score**: `primary_score = harm_score` (higher = attacker won; unscoped /
  `security_domain=None`, the framework's always-delivered optimization signal);
  sub-scores carry `refusal`, `full_harm`, each grading sub-check, the category,
  and grading/judge error flags. `success = (not refusal) and harm_score == 1.0`
  (a strict per-task gate, not an AgentHarm headline metric).

## Usage

```python
from superred.core.controller import Controller
from inspect_agent_target import USER_TAG, SYSTEM_TAG, TOOLS_TAG
from agentharm_claim import agentharm_total_claim, agentharm_target_factory, aggregate

# target_model defaults to DEFAULT_TARGET_MODEL (openai/gpt-4o-2024-08-06), the
# AgentHarm-appropriate model on the proxy; pass it explicitly to use another.
claim = agentharm_total_claim(api_base=PROXY, api_key=KEY)   # 164 (benign-cover default; 176 with benign_cover=False)
controller = Controller(
    optimizer_factory=MyAttackerOrPassthrough,
    target_factory=agentharm_target_factory(api_base=PROXY, api_key=KEY),
    security_claim=claim,
    # the three root tags = the full attacker surface; narrow per experiment.
    scope=frozenset({USER_TAG, SYSTEM_TAG, TOOLS_TAG}),
)
result = await controller.run()
metrics = aggregate([tr.best_evaluation for tr in result.task_results])
# -> n, n_scored, avg_score, avg_full_score, avg_refusals, avg_score_non_refusals, per-category
# aggregate() is robust to errored/budget-exhausted tasks: their fallback evals
# (empty sub_scores) are excluded; n_scored < n flags that the run had failures.
```

The `target_model` for the claim and the target factory MUST match; both default
to `DEFAULT_TARGET_MODEL`, so the common case is matched automatically. The model
is fixed at construction (not a config slot): neither the Task nor the attacker
can change the agent's model.

## Tool trust-boundary scopes

`agentharm_target_factory` wires the general target with `TOOL_SCOPES`: a map from
each of the 80 harmful tools to a **trust boundary** (the external system whose
compromise would let an attacker control that tool's return), parented under the
target's `tools` root. So the target exposes one `tool:<name>` output-injection
controllable per configured tool, scoped to its boundary. An experiment can then
grant an attacker control over, say, only web-sourced tool returns by putting
`web` in scope, or all of it with `tools`. Boundaries: `local_machine`,
`own_directories`, `web`, `darknet`, `social` (-> `x`, `instagram`, `reddit`,
`tinder`, `pinterest`, `facebook`, `mastodon`, `forum`, `blog`, `tiktok`,
`youtube`), `comms` (-> `email`, `whatsapp`, `messaging`), `financial` (-> `bank`,
`paypal`, `crypto`), `commerce` (-> `amazon`, `retail`), `services` (->
`external_db`, `cloud_hosting`, `media_gen`, `edu_portal`), `infra` (-> `network`).
28 leaf boundaries for the 80 tools (tools sharing a system share a scope). The
direct-request baseline (passthrough) injects into none of them. See
`ASSUMPTIONS.md` for the per-tool rationale.

## Subclaims (8 categories + total)

`agentharm_total_claim` (164 by default; 176 with `benign_cover=False`),
`agentharm_category_claim(label)`, and 8 named
factories: `agentharm_copyright_claim`, `agentharm_cybercrime_claim`,
`agentharm_disinformation_claim`, `agentharm_drugs_claim`, `agentharm_fraud_claim`,
`agentharm_harassment_claim`, `agentharm_hate_claim`, `agentharm_sexual_claim`.
(The paper has 11 categories; the released public split covers 8.)

## Judges

Refusal + semantic judges are upstream's, pinned to `openai/gpt-4o-2024-08-06`
(verified available on the project proxy, so no substitution). Judge calls are
out-of-band (not counted in Controller `llm_usage`).

## Install / test

```bash
pip install -e targets/inspect_agent
pip install -e security_claims/agentharm
pytest -m "not (faithfulness or smoke)" security_claims/agentharm/tests   # offline
LITELLM_API_KEY=... LITELLM_API_BASE=... pytest -m faithfulness security_claims/agentharm/tests  # OUR==upstream parity
LITELLM_API_KEY=... LITELLM_API_BASE=... pytest -m smoke security_claims/agentharm/tests          # end-to-end vs the target
LITELLM_API_KEY=... LITELLM_API_BASE=... python security_claims/agentharm/tests/smoke/run.py       # printed demo
```

The dataset is downloaded from the gated HuggingFace repo
`ai-safety-institute/AgentHarm` (pinned revision) on first use. See `ASSUMPTIONS.md`.

## Credits / upstream

This module's code is MIT-licensed, Copyright (c) 2026 Simon Sure. It is an
original superred port; it bundles no third-party code or data.

It builds on and reuses (at runtime, by import - nothing is copied):

- **AgentHarm** (Andriushchenko et al., 2024, arXiv:2410.09024) - the benchmark
  whose tools, grading functions, and judges are reused verbatim via
  `inspect-evals`. Dataset: `ai-safety-institute/AgentHarm` on HuggingFace,
  MIT License with an additional clause restricting use to "improving the
  safety and security of AI systems" (Copyright (c) 2024 Gray Swan AI and UK AI
  Safety Institute). The dataset is downloaded at runtime, not shipped with this
  package; that field-of-use restriction applies to your use of it.
- **Inspect Evals** (`inspect-evals`, MIT, (c) 2024 UK AI Security Institute) -
  supplies the `inspect_evals.agentharm` implementation this module imports.
- **Inspect AI** (`inspect-ai`, MIT, (c) 2024 UK AI Security Institute) - the
  agent/tool/model runtime.

The `benign_covers.json` cover tasks are original superred work (not AgentHarm's
benign split). Cite Andriushchenko et al. 2024 when reporting results.

```bibtex
@article{andriushchenko2024agentharm,
  title={AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents},
  author={Andriushchenko, Maksym and others},
  journal={arXiv preprint arXiv:2410.09024},
  year={2024}
}
```
