Metadata-Version: 2.4
Name: agentic-security-harness
Version: 1.14.0
Summary: Open-source defensive harness and learning lab for reproducing and measuring agentic AI failure modes with portable traces, attack graphs, scorecards, and data-boundary tests.
Author: Dmitry Krivonosov
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/krivonosoff161/agentic-security-harness
Project-URL: Repository, https://github.com/krivonosoff161/agentic-security-harness
Project-URL: Issues, https://github.com/krivonosoff161/agentic-security-harness/issues
Project-URL: Changelog, https://github.com/krivonosoff161/agentic-security-harness/blob/main/CHANGELOG.md
Keywords: ai-security,agentic-ai,benchmark,llm-security,prompt-injection,red-team
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: pydantic<3,>=2
Provides-Extra: document-agent
Requires-Dist: pydantic-ai-slim==1.107.7; extra == "document-agent"
Provides-Extra: transfer
Requires-Dist: agentic-transfer-verifier==0.2.1; extra == "transfer"
Requires-Dist: agentic-transfer-verifier-harness-extension==1.0.1; extra == "transfer"
Provides-Extra: handoff
Requires-Dist: ai-agent-handoff==0.3.0; extra == "handoff"
Requires-Dist: ai-agent-handoff-harness-extension==1.0.0; extra == "handoff"
Provides-Extra: playbooks
Requires-Dist: llm-safety-playbooks==0.1.0; extra == "playbooks"
Provides-Extra: router
Requires-Dist: agentic-llm-router==0.2.1; extra == "router"
Provides-Extra: filter
Requires-Dist: llm-cheap-filter==0.2.0; extra == "filter"
Provides-Extra: all
Requires-Dist: agentic-transfer-verifier==0.2.1; extra == "all"
Requires-Dist: agentic-transfer-verifier-harness-extension==1.0.1; extra == "all"
Requires-Dist: ai-agent-handoff==0.3.0; extra == "all"
Requires-Dist: ai-agent-handoff-harness-extension==1.0.0; extra == "all"
Requires-Dist: llm-safety-playbooks==0.1.0; extra == "all"
Requires-Dist: agentic-llm-router==0.2.1; extra == "all"
Requires-Dist: llm-cheap-filter==0.2.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Requires-Dist: mypy>=1.8; extra == "dev"
Dynamic: license-file

# Agentic Security Harness

Distribution version: **1.14.0**. Python 3.11 or newer. Apache-2.0.

A local defensive benchmark and toolkit for testing agent boundary failures:
untrusted input, authority confusion, retained history and controlled file actions.
It produces replayable traces, scorecards and separately verifiable evidence.

This description belongs to the distribution you are viewing. For live publication
status, exact artifact hashes and verification runs, see the
[release records](https://github.com/krivonosoff161/agentic-security-harness/releases)
and [current project state](https://github.com/krivonosoff161/agentic-security-harness/blob/main/docs/current-state.md).
Version metadata alone does not attest index availability or publication.

## Install this version

```bash
python -m pip install agentic-security-harness==1.14.0
ash quickstart --out reports/quickstart
```

Use a fresh output directory. The quickstart runs fixed offline scenarios against
vulnerable and protected fixtures; it does not call a model or an external target.

## Check a controlled file boundary

```bash
ash controlled-file-workflow --out reports/file-boundary
ash controlled-file-verify --out reports/file-boundary
```

This opt-in workflow performs actual writes to newly created synthetic files.
Report-only Runtime Guard authority permits the report file and denies the
protected file. A separate read-only verifier checks disk bytes and captured
history. Model output cannot supply a filesystem path or executable code.
The default run is offline. An existing local Ollama model can be selected
explicitly using the documented options; installing this package starts no model.

[Versioned instructions and limits](https://github.com/krivonosoff161/agentic-security-harness/blob/v1.14.0/docs/controlled-file-workflow.md)
explain the guarded/ablated fixed control separately from model observations.
Permission enforcement and correctness of the generated report are different
measurements. This is not an arbitrary tool executor or a hostile-code sandbox.

This version also includes `ControlledFileSession`, a narrow fresh-fixture hook for
trusted applications to submit proposal bytes through the same boundary. An
optional repository example connects that hook to a Pydantic AI tool callback;
Pydantic AI is not a base dependency. The application owns context, labels and
files, and the model supplies only proposal text. Storage failures retain an
incomplete directory for inspection; they do not undo earlier completed writes.

## Save a configured text document

The [guarded workspace writer](https://github.com/krivonosoff161/agentic-security-harness/blob/v1.14.0/docs/guarded-workspace-writer.md)
lets a trusted host configure an output directory and aliases for new UTF-8 files.
An agent proposes `write_text` with an alias and document content. The host's
deterministic Runtime Guard decides whether the proposal may create that file.
The create-only writer reads back the output and records content-free intent
and result receipts; an existing file is preserved.

Use `WorkspacePolicy` and `GuardedWorkspace` as an application tool, or the
installed `ash workspace-check`, `ash workspace-write` and
`ash workspace-verify` commands. The optional `ash workspace-run --execute`
makes one request to an existing local Ollama service. No model call is needed
for the Guard decision. The host owns paths, policy and classification; proposed
text cannot select arbitrary paths or label its own sensitivity. This in-process
tool is not an OS sandbox or a semantic checker of the saved document.

## Run separate document jobs

The [document workflow](https://github.com/krivonosoff161/agentic-security-harness/blob/v1.14.0/docs/document-workflow.md)
adds host configuration, a no-effect preview, an explicit one-generation job,
read-only status and an optional Pydantic AI bridge. A later job may use a
verified previous output as untrusted source text, never as inherited authority.
The host still chooses the output policy and any exact JSON or checklist
requirements. Quality status describes only declared deterministic checks;
even `checked` does not certify meaning or factual accuracy. An invalid-format
document remains saved as a draft but is not reusable as a chain input.

This version also binds document generation to a host-selected content-only
tool: model text cannot choose an output alias, path or authority. Reusing a
review-required document needs an exact-byte host review digest; a failed
quality check still blocks reuse. Hosts can bind input restrictions to source
bytes and original expiry times, including bounded multi-source composition.
Fixed plans and dynamic admissions declare expected jobs and pending choices;
coverage reports whether declared history is accounted for, not whether the
documents are factually right. Optional supervised execution may fence an
interrupted owned child and permit exact reviewed bytes as data for a **new**
job; the original job remains unresolved and cannot be replayed. No base
model-framework dependency, arbitrary-path tool or OS sandbox is added.
The [versioned operator and replay guide](https://github.com/krivonosoff161/agentic-security-harness/blob/v1.14.0/docs/document-workflow-replay.md)
separates scripted transport, real-model use and human review.

## Admit captured sources and reconcile an interrupted write

The host-owned source-capture API joins source restrictions, captured observations
and independently retained host expectations before admitting source bytes. Its
installed two-step example checks task quality before passing one output to the
next task. Admission proves agreement with the declared capture contract, not
the truth of a source or completeness of uninstrumented host activity.

A separate create-only workspace operation coordinator retains consumed grants
and fences supported old deliveries. After interruption, it can reconcile exact
file bytes as a postcondition, or verify absence before an explicitly authorized
new attempt. It cannot infer that the original call returned successfully.
Partial or mismatching output remains unknown and is preserved. These guarantees
depend on trusted host anchors, the supported locking filesystem and exclusive
participation in the coordinator; they do not provide general exactly-once tools.

## Other included surfaces

- Portable trace validation, scorecards and deterministic comparison reports.
- Retained ancestry admission and the closed synthetic Runtime Gateway.
- An experimental single-plan garak adapter, not an official NVIDIA integration
  or support for every garak detector.
- Separately versioned optional Transfer, Handoff, Playbooks, Router and Filter
  packages. Installation does not automatically activate them.

## Evidence and scope

The earlier eight-call observation is historical, and this version does not
turn it into a new package-specific security-effectiveness claim.
The historical eight-call observation used a specific installed 1.9.0 candidate:
four protected proposals were denied, four reports were written, and two reports
were correct. It is not a measurement of this version or a reliability estimate.

A separate six-call local Qwen2.5 1.5B integration observation for this hook wrote
four reports, of which two had correct counts, and denied two protected writes.
The eight fixed seed cases and separate offline timing runs are different evidence
classes. None establishes cross-model reliability or production containment.

[Historical observation](https://github.com/krivonosoff161/agentic-security-harness/blob/v1.9.0/docs/controlled-file-observation-20261001.md)
and [methodology](https://github.com/krivonosoff161/agentic-security-harness/blob/v1.14.0/docs/benchmark-semantics.md)
separate controlled evidence from broader claims. Production-wide containment,
arbitrary host protection and independent human review are not established.

Author: Dmitry Krivonosov ([krivonosoff161](https://github.com/krivonosoff161)).
Questions and reproducible reports: [GitHub issues](https://github.com/krivonosoff161/agentic-security-harness/issues).
