Metadata-Version: 2.5
Name: readiness-kit
Version: 0.1.1
Summary: The open-source frame under the Agent Readiness Gate: an evaluation harness with a held-out split, a starter attack pack mapped to OWASP ASI01-ASI10, adapters for the open agent scanners, an eight-area readiness scorecard, a CycloneDX Agent BOM emitter and a cost-per-task baseline.
Project-URL: Homepage, https://rainkernel.com/open-source
Project-URL: Documentation, https://github.com/rainkernel/readiness-kit/tree/main/docs
Project-URL: Repository, https://github.com/rainkernel/readiness-kit
Project-URL: Issues, https://github.com/rainkernel/readiness-kit/issues
Project-URL: Changelog, https://github.com/rainkernel/readiness-kit/blob/main/CHANGELOG.md
Project-URL: Sample Evidence Report, https://rainkernel.com/products/agent-readiness-gate/sample-report
Author-email: Rainkernel Technologies Private Limited <oss@rainkernel.com>
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: ai-agents,cyclonedx,evaluation,llm,mcp,owasp,readiness,sbom,security
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Requires-Dist: httpx>=0.27
Requires-Dist: jsonschema>=4.18
Requires-Dist: pyyaml>=6.0
Provides-Extra: benchmark
Requires-Dist: datasets>=2.19; extra == 'benchmark'
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Description-Content-Type: text/markdown

# Readiness Kit

The frame under the [Agent Readiness Gate](https://rainkernel.com/products/agent-readiness-gate), in public:
a command-line tool that runs an evaluation set against an AI agent and scores it on a held-out split, runs a
starter attack pack mapped to the OWASP Top 10 for Agentic Applications, orchestrates the open agent scanners,
emits a CycloneDX Agent Bill of Materials, computes a cost-per-task baseline from traces, and applies the
eight-area readiness rubric to all of it. The engine (the full attack pack, the domain packs, the seed sets, the
Evidence Report generator and the CI gate) ships in the licensed Gate.

**Release: v0.1.1 · 22 October 2026.** Apache-2.0. Python 3.10+.

```
pip install readiness-kit
rk demo
```

`rk demo` runs the whole pipeline on a bundled, deliberately weak support-ticket agent. It works offline, needs
no model and no key, and prints a scorecard in about a second. Then `rk init` and point `rk.yaml` at your own
agent: [the ten-minute walkthrough](https://github.com/rainkernel/readiness-kit/blob/main/docs/WALKTHROUGH.md).

## What it does

| Command | What it measures |
| --- | --- |
| `rk split SET` | Makes the held-out split, `sha256(salt + ":" + id) < fraction`, so the headline number is on cases the agent was not tuned on, and anyone with the salt reproduces it |
| `rk eval` | Runs every case against the agent and grades it with ten deterministic graders; pass rate held-out and dev, by category, with the set's hash in the result |
| `rk attack` | Runs the attack pack: twelve starter cases over ASI01 to ASI10, one deterministic oracle set each. Reports which cases the agent followed, by class and severity |
| `rk scan` | Runs the open scanners you have installed (Cisco mcp-scanner, Snyk agent-scan, SPLX Agentic Radar, Promptfoo), keeps their raw output and normalises the findings; a skipped scanner is named with the reason |
| `rk bom` | Emits the Agent BOM (CycloneDX 1.6) from `agent.yaml` plus what it discovers on disk (MCP configs, skill files, prompts) and flags everything unversioned or unpinned |
| `rk cost` | Cost per completed task from OpenTelemetry, OTLP or Langfuse traces, or from the Kit's own run: tokens, calls, retries; against a ceiling; projected to volume |
| `rk score` | Applies the rubric (eight areas, weights summing to 100) to the artefacts above and the declared assessment; every area says whether it was measured or declared |

Every command writes a JSON artefact into `runs/` with the hashes and versions a published number must cite;
`rk score` writes the scorecard as JSON and Markdown. The licensed Gate consumes these artefacts unchanged.

## How an agent is wired in

One small contract (docs/FORMATS.md): the Kit sends `{case_id, input, context, metadata}` and reads
`{output, tool_calls, steps, usage}`; only `output` is required. Five target types: `python` (a function
in-process), `http` (an endpoint you control), `openai-chat` (any OpenAI-compatible chat endpoint with your
system prompt), `command` (stdin to stdout), `replay` (a transcript). What an agent does not report, the Kit
marks *not observable*. Silence is not upgraded to a pass.

## Repository

| Folder | What it holds |
| --- | --- |
| `src/readiness_kit/harness/` | Targets, evaluation sets, the split, graders, the runner, the attack runner |
| `attack-pack/` | The starter pack (`starter/asi01…asi10.yaml`, twelve cases) and the case schema, in the format the full pack uses |
| `rubric/` | `readiness-rubric.yaml` (areas, weights, bands, scoring rules) and the schemas of the scorecard and the Evidence Report |
| `src/readiness_kit/bom/` | Manifest, discovery, CycloneDX 1.6 emitter |
| `src/readiness_kit/cost/` | Trace parsers (OTel JSONL, OTLP JSON, Langfuse, rk), price table, baseline and projection |
| `src/readiness_kit/adapters/` | The four scanner adapters and the adapter contract |
| `examples/ticket-triage-agent/` | The demo agent, its 80-case set, traces, manifest, assessment and example price table |
| `benchmarks/bitext-support/` | The published benchmark: data preparation, configuration, the one command |
| `docs/` | [DESIGN](https://github.com/rainkernel/readiness-kit/blob/main/docs/DESIGN.md) · [WALKTHROUGH](https://github.com/rainkernel/readiness-kit/blob/main/docs/WALKTHROUGH.md) · [FORMATS](https://github.com/rainkernel/readiness-kit/blob/main/docs/FORMATS.md) · [ADAPTERS](https://github.com/rainkernel/readiness-kit/blob/main/docs/ADAPTERS.md) · [BENCHMARK](https://github.com/rainkernel/readiness-kit/blob/main/docs/BENCHMARK.md) |

A worked example of what the Gate produces from these artefacts, the Evidence Report on a fictional
claims-triage agent, is at
[rainkernel.com/products/agent-readiness-gate/sample-report](https://rainkernel.com/products/agent-readiness-gate/sample-report).

## Principles

**Frame open, engine licensed.** The harness, the starter pack, the rubric, the BOM emitter, the cost baseline
and the adapters are public; the full attack pack, the domain packs, the seed sets, the report generator and the
CI gate are what a licence buys. **Honest version numbers.** v0.1 means v0.1, and no release is announced before
it is tagged. **Nothing moves behind a paywall.** What ships in the open tier stays there. **Upstream first.**
Where an open scanner should have a feature, we contribute it there before wrapping it here. The full note:
[docs/DESIGN.md](https://github.com/rainkernel/readiness-kit/blob/main/docs/DESIGN.md).

## Releases

- **v0.1.1 · 22 October 2026.** The README's links are absolute, so they work on PyPI as well as on GitHub (on PyPI the walkthrough link returned 404). No code change.
- **v0.1.0 · 22 October 2026** (planned for the 18th, four days late; the date is recorded, not rewritten). The frame above, with the benchmark on a public support-ticket set
  (docs/BENCHMARK.md) and the ten-minute walkthrough.
- Later releases are announced when they are tagged, here and in the [Lab's log](https://rainkernel.com/lab),
  and not before. CHANGELOG.md has the detail.

## Developing

```
git clone https://github.com/rainkernel/readiness-kit && cd readiness-kit
pip install -e ".[dev]"
pytest -q && ruff check src tests examples
```

CONTRIBUTING.md has the rest: issues first, DCO sign-off, what belongs in the Kit and what belongs in the Gate.
RELEASING.md is the release checklist.

## Security

What the Kit itself does and does not do: no telemetry and no calls home (the only network traffic is to the
target you configure and whatever the scanners you install make themselves); scanners run as subprocesses with
explicit arguments and no shell; secrets are read from environment variables you name in rk.yaml, and a missing
one stops the run before it starts; attack canaries sit on `.invalid` domains that cannot resolve. Run the
attack pack against a staging agent with outbound tools stubbed: the cases are real attacks and your agent may
act on them ([docs/WALKTHROUGH.md](https://github.com/rainkernel/readiness-kit/blob/main/docs/WALKTHROUGH.md)).

Vulnerabilities in this tool: security@rainkernel.com or a private security advisory on this repository. We
credit reporters. See [SECURITY.md](https://github.com/rainkernel/readiness-kit/blob/main/SECURITY.md) and
[rainkernel.com/.well-known/security.txt](https://rainkernel.com/.well-known/security.txt).

## Licence

Apache-2.0. See [LICENSE](https://github.com/rainkernel/readiness-kit/blob/main/LICENSE) and [NOTICE](https://github.com/rainkernel/readiness-kit/blob/main/NOTICE). © 2026 Rainkernel Technologies Private Limited.
