Metadata-Version: 2.4
Name: evalopt-graph
Version: 0.1.1
Summary: Deterministic acceptance policy for AI coding agents: evaluates host-supplied gate results, claims and hash-bound evidence and returns a replayable ACCEPTED, BLOCKED, FAILED, UNSUPPORTED or UNVERIFIED decision. Zero runtime dependencies, Python 3.10-3.14.
Project-URL: Homepage, https://github.com/ajaysurya1221/evalopt-graph
Project-URL: Documentation, https://github.com/ajaysurya1221/evalopt-graph#readme
Project-URL: Repository, https://github.com/ajaysurya1221/evalopt-graph
Project-URL: Issues, https://github.com/ajaysurya1221/evalopt-graph/issues
Project-URL: Changelog, https://github.com/ajaysurya1221/evalopt-graph/blob/main/CHANGELOG.md
Author: Ajay Surya Senthilrajan
Maintainer: Ajay Surya Senthilrajan
License-Expression: MIT
License-File: LICENSE
Keywords: agents,evaluation,evidence,governance,reproducibility
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Provides-Extra: all
Requires-Dist: anthropic>=1.11.0; extra == 'all'
Requires-Dist: jsonschema<5,>=4.26.0; extra == 'all'
Requires-Dist: openai>=3.23.0; extra == 'all'
Requires-Dist: pytest>=9.1.1; extra == 'all'
Requires-Dist: pyyaml>=6; extra == 'all'
Requires-Dist: ruff>=0.16.10; extra == 'all'
Provides-Extra: dev
Requires-Dist: jsonschema<5,>=4.26.0; extra == 'dev'
Requires-Dist: pytest>=9.1.1; extra == 'dev'
Requires-Dist: ruff>=0.16.10; extra == 'dev'
Provides-Extra: llm
Requires-Dist: anthropic>=1.11.0; extra == 'llm'
Requires-Dist: openai>=3.23.0; extra == 'llm'
Provides-Extra: yaml
Requires-Dist: pyyaml>=6; extra == 'yaml'
Description-Content-Type: text/markdown

# evalopt-graph

> Standalone feature development has ended. This repository preserves the acceptance kernel and its research record. For new systems, use the alternatives listed below. Existing evidence remains subject to its original versions and limitations. Evalopt evaluates supplied evidence against policy; it is not a semantic truth oracle.
>
> [Alternatives](https://github.com/ajaysurya1221/evalopt-graph#alternatives) · [Migration map](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/migration.md) · [Portfolio index](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/portfolio.md) · [Why](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/decisions/archive-standalone.md)

**Make acceptance policy explicit and replayable.**

Your harness runs the checks and supplies observations.
Evalopt applies the policy and returns a decision with stable reason codes.

**Expected output from the example below, using authored inputs:**
```text
ACCEPTED ('policy_satisfied',)
FAILED ('required_gate_failed:tests',)
BLOCKED ('tests_weakened',)
```
[Run the example](https://github.com/ajaysurya1221/evalopt-graph#decide-serialize-replay) · [Inspect the kernel](https://github.com/ajaysurya1221/evalopt-graph/blob/main/src/evalopt_graph/kernel.py)

Five outcomes: ACCEPTED, BLOCKED, FAILED, UNSUPPORTED and UNVERIFIED.

**Boundary:** the host runs checks, detects weakened tests and supplies trustworthy evidence.
Replay establishes consistency with that input, not the truth of the input.

**Engineering:** [Zero runtime dependencies](https://github.com/ajaysurya1221/evalopt-graph/blob/main/pyproject.toml) · [State and replay tests](https://github.com/ajaysurya1221/evalopt-graph/blob/main/tests/test_kernel.py)
[32 authored/generated conformance cases](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/BENCHMARK_RESULTS.md) · [Release checks](https://github.com/ajaysurya1221/evalopt-graph/blob/main/scripts/verify_installed_release.py)

Conformance evidence only; no external capability or comparative result is claimed.
[Install](https://github.com/ajaysurya1221/evalopt-graph#install) · [Host responsibilities](https://github.com/ajaysurya1221/evalopt-graph#kernel-owns--host-owns)

<picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://github.com/ajaysurya1221/evalopt-graph/raw/main/docs/assets/hero-dark.svg">
  <img src="https://github.com/ajaysurya1221/evalopt-graph/raw/main/docs/assets/hero-light.svg" alt="evalopt-graph, acceptance policy kernel. Do the gates, claims and evidence satisfy the acceptance policy? Evidence card from the README example with required gates tests and lint: tests PASS and lint PASS gives ACCEPTED ('policy_satisfied',); tests FAIL gives FAILED ('required_gate_failed:tests',); passing gates with a host-reported weakened test suite give BLOCKED ('tests_weakened',)." width="100%">
</picture>

[![Release](https://img.shields.io/github/v/release/ajaysurya1221/evalopt-graph?display_name=tag&sort=semver)](https://github.com/ajaysurya1221/evalopt-graph/releases/latest) [![CI](https://github.com/ajaysurya1221/evalopt-graph/actions/workflows/ci.yml/badge.svg)](https://github.com/ajaysurya1221/evalopt-graph/actions/workflows/ci.yml) [![PyPI](https://img.shields.io/pypi/v/evalopt-graph?cacheSeconds=300)](https://pypi.org/project/evalopt-graph/) [![Python](https://img.shields.io/pypi/pyversions/evalopt-graph?cacheSeconds=300)](https://pypi.org/project/evalopt-graph/) [![License: MIT](https://img.shields.io/badge/license-MIT-49C6B8.svg)](https://github.com/ajaysurya1221/evalopt-graph/blob/main/LICENSE)

> **Apply acceptance policy to your CI observations**
>
> Have the host report its test and lint results through `AcceptanceInput`. Evaluate them against `GovernancePolicy`, save the decision with its policy and input, then deserialize and replay it later. The example shows passing gates, a failed test gate, and a host-reported weakened test suite producing different outcomes.
>
> This workflow is exercised with authored inputs. Evalopt neither runs those checks nor discovers weakened tests itself.

## Alternatives

New systems should start from one of these destinations. Seven jobs, one destination each, in the
order fixed by the [migration map](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/migration.md). These are job-oriented
pointers, not tested drop-in replacements. The four sibling projects are independent of evalopt and
of each other. This guide and the sources it inspected supply no automatic or lossless mapping of
evalopt's five decision states, reason codes or decision records into any destination. A content
hash binds bytes; a trusted signature binds a statement to an identity within its verification
system; neither proves a claim true.

| Your job | Go to | What it does, in its own terms |
| --- | --- | --- |
| General policy evaluation | [OPA](https://www.openpolicyagent.org/docs/philosophy) | Evaluates policies over structured data and query input and returns structured decisions. |
| Software provenance and attestation verification | [in-toto attestation framework](https://github.com/in-toto/attestation) and [Witness](https://github.com/in-toto/witness) | A Statement binds a typed predicate to artifact digests; a DSSE envelope carries signatures; Witness verifies signed attestation collections against a signed policy. |
| Native merge enforcement | [GitHub rulesets and required checks](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets) | Applies required status checks and bypass permissions to targeted branches; check states are set by write-capable producers. |
| Action-policy qualification | [Actseal](https://github.com/ajaysurya1221/actseal) | Tests a model's action policy under a frozen policy against recorded evaluation evidence and replays it offline; the application owns execution. |
| Code-claim warrants | [Dorian](https://github.com/ajaysurya1221/dorian) | Turns explicit code claims into checks sealed beside the code and revokes the warrant when a load-bearing check fails. |
| PR scope | [Frontier Scout](https://github.com/ajaysurya1221/frontier-scout) | Checks a diff against the base commit's scope policy and compiles the same policy into Claude Code permissions and hooks; a research preview. |
| Stochastic regression | [ARCI](https://github.com/ajaysurya1221/agent-reliability-ci) | Freezes a repeated experiment, gates a change on relative non-inferiority and exports a replayable failure; its hashing helpers were adapted from evalopt with attribution, not as a dependency. |

The [migration map](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/migration.md) covers each of the ten stable exports
and six cross-cutting topics with its own limit, and the
[portfolio index](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/portfolio.md) describes the four sibling projects.
The deprecated compatibility surfaces (graph, CLI, providers, Codex bridge, research loop and
adapters) have no supported compatible migration target in this map; see entry 16.

## Install

Install into an isolated environment (Python 3.10–3.14), then run the example below:

```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install evalopt-graph==0.1.1
```

On Windows PowerShell, activate with `.venv\Scripts\Activate.ps1`.

## Decide, serialize, replay

```python
from evalopt_graph import AcceptanceDecision, AcceptanceInput, GovernancePolicy, evaluate_acceptance

policy = GovernancePolicy(required_gates=("tests", "lint"))

# What the harness observed: both required gates passed, nothing was skipped or deleted.
observed = AcceptanceInput(
    observed_at="2026-07-20T12:00:00+00:00",
    gate_results=(("tests", "PASS"), ("lint", "PASS")),
)
decision = evaluate_acceptance(policy, observed)
print(decision.status, decision.reasons)  # ACCEPTED ('policy_satisfied',)

# Same policy, but the test gate failed.
failed = evaluate_acceptance(
    policy,
    AcceptanceInput(
        observed_at="2026-07-20T12:00:00+00:00",
        gate_results=(("tests", "FAIL"), ("lint", "PASS")),
    ),
)
print(failed.status, failed.reasons)  # FAILED ('required_gate_failed:tests',)

# The host reports passing gates and a weakened test suite.
blocked = evaluate_acceptance(
    policy,
    AcceptanceInput(
        observed_at="2026-07-20T12:00:00+00:00",
        gate_results=(("tests", "PASS"), ("lint", "PASS")),
        tests_weakened=True,
    ),
)
print(blocked.status, blocked.reasons)  # BLOCKED ('tests_weakened',)

# Decisions are plain, content-addressed records: store them, then re-check them later.
restored = AcceptanceDecision.from_dict(decision.to_dict())
assert restored.validate()
assert restored.replay(policy, observed)
```

The evaluator is pure, with no LLM in the loop: the same policy and input produce the same decision,
with stable reason codes and hashes for the policy, input, evidence records, and decision. Its
[precedence](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/architecture.md#acceptance-precedence) is fail-closed: malformed or tampered input
`FAILED`, policy or contradiction block `BLOCKED`, observed gate failure `FAILED`, unavailable required
mechanism `UNSUPPORTED`, incomplete required evidence `UNVERIFIED`, otherwise `ACCEPTED`.

## Stable kernel API

The package root exports exactly ten stable symbols in `v0.1.x`: `GovernancePolicy`, `ClaimRecord`,
`EvidenceRequest`, `EvidenceMaterial`, `EvidenceAttestation`, `SupportAssessment`, `EvidenceAuthority`,
`AcceptanceInput`, `AcceptanceDecision` and `evaluate_acceptance`.

- `GovernancePolicy` declares required gates, trust, freshness, evaluator, claim, verifier, and record
  authorization requirements.
- `EvidenceAuthority` applies controller-owned adapter policy to material already retrieved by a host.
- `EvidenceAttestation` binds source policy, retrieval metadata, selected content, and hashes.
- `SupportAssessment` separately records whether selected content supports, contradicts, or is
  insufficient for a claim.
- `evaluate_acceptance` evaluates a complete immutable observation once and returns a replayable
  `AcceptanceDecision`.

See [`kernel.py`](https://github.com/ajaysurya1221/evalopt-graph/blob/main/src/evalopt_graph/kernel.py) for the compact public contract and
[`test_kernel.py`](https://github.com/ajaysurya1221/evalopt-graph/blob/main/tests/test_kernel.py) for complete evidence-authority examples.

## Kernel owns / host owns

<picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://github.com/ajaysurya1221/evalopt-graph/raw/main/docs/assets/where-dark.svg">
  <img src="https://github.com/ajaysurya1221/evalopt-graph/raw/main/docs/assets/where-light.svg" alt="Two lanes. Host lane: your host runs the checks, detects weakened tests, collects evidence, holds the GovernancePolicy and builds an AcceptanceInput. Kernel lane: evaluate_acceptance returns an AcceptanceDecision in one of five states: ACCEPTED, BLOCKED, FAILED, UNSUPPORTED or UNVERIFIED. The host stores policy, input and decision; replay later recomputes the decision and is true only for an identical record." width="100%">
</picture>

In words: your host runs the checks, detects weakened tests, collects evidence and builds an `AcceptanceInput`.
`evaluate_acceptance(policy, input)` returns an `AcceptanceDecision`. The host stores policy, input and decision;
`replay()` later recomputes the decision and compares the record.

| The kernel owns | Your host owns |
| --- | --- |
| Deterministic policy evaluation | Model calls, prompts, and sampling |
| Terminal decision status and reason codes | Tools, edits, retries, and orchestration |
| Content-bound evidence and support records | Retrieval and protection of controller policy |
| Record authorization and freshness checks | Sandboxes, network and filesystem controls |
| Canonical serialization, hashes, and replay | Persistence, scheduling, graders, and metrics |

This boundary is intentional. Importing `evalopt_graph` loads the kernel without importing provider
SDKs, the compatibility graph, CLI code, filesystem adapters, benchmark adapters, or optional
frameworks.

## Evidence, not a truth oracle

Evalopt keeps three questions separate:

1. **Was this exact material retrieved under an allowed adapter policy?** An attestation binds the
   retrieved bytes, selected span, source class, time, parser identity, and policy.
2. **Does the selected material support this exact claim?** A support assessment records the verifier
   and its result independently of retrieval integrity.
3. **May these exact records authorize acceptance?** Governance policy allowlists the authority policy,
   verifier, and evidence records before the decision can use them.

That structure makes evidence tampering and unsupported load-bearing claims visible. It does **not**
prove semantic truth, deployed behavior, source correctness, or model capability.

### Current evidence ladder

- **IMPLEMENTED / UNIT_PROVEN:** pure API, immutable records, distinct terminal states, authorization,
  serialization, replay, and import isolation.
- **INTEGRATION_PROVEN:** the compatibility host and optional Harbor verifier mapping terminate through
  the same kernel decision path.
- **CONFORMANCE_PROVEN:** deterministic generated and authored cases exercise a bounded evidence
  boundary mechanism.
- **What this does not claim:** external benchmark results, comparison against other tools, or
  independent reproduction. The conformance cases show the mechanism behaves as specified; they say
  nothing about model capability.

Read the [evidence report](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/BENCHMARK_RESULTS.md) and
[external evaluation protocol](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/BENCHMARK_PROTOCOL.md) for the claims, controls, and gaps.
The protocol is preserved as a historical document; no new benchmark programme runs.
There is no live-model governance campaign or official Docker SWE-bench result in this release.

## Historical integration examples

The kernel was designed to accept host-supplied observations through `AcceptanceInput`, so that a
host could adapt observations from Codex, Claude Code, OpenHands, LangGraph, CI or its own runtime
into that input and store the returned decision beside the policy and input used to create it. The
host remains responsible for both steps. The examples below are preserved as they were. They
are not an invitation to build new evalopt integrations; new systems should start from the
[alternatives](https://github.com/ajaysurya1221/evalopt-graph#alternatives) above.

An optional
[Harbor 0.18 conformance task](https://github.com/ajaysurya1221/evalopt-graph/blob/main/bench/harbor/README.md)
demonstrates verifier-side mapping without importing Harbor into the kernel. It validates integration
wiring only; it is not an external benchmark.

The historical graph, standalone CLI, provider clients, Codex bridge, research loop, and filesystem
adapters remain as **deprecated compatibility surfaces**. They receive no feature work, are not stable
API or policy authorities, and have no supported compatible migration target in this map
([entry 16 of the migration map](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/migration.md#the-sixteen-entries)).
Anything that still imports evalopt should depend only on the ten root exports above.

## Documentation

- [Migration map: where each evalopt concept goes](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/migration.md)
- [Portfolio index: the four independent sibling projects](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/portfolio.md)
- [Wind-down decisions](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/decisions/archive-standalone.md)
- [Architecture and trust boundaries](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/architecture.md)
- [Evidence results and limitations](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/BENCHMARK_RESULTS.md)
- [External evaluation protocol (historical, preserved; no new benchmark programme)](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/BENCHMARK_PROTOCOL.md)
- [Harbor integration](https://github.com/ajaysurya1221/evalopt-graph/blob/main/bench/harbor/README.md)
- [v0.1.1 final deprecation release notes](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/releases/v0.1.1.md)
- [v0.1.0 release notes](https://github.com/ajaysurya1221/evalopt-graph/blob/main/docs/releases/v0.1.0.md)
- [Changelog](https://github.com/ajaysurya1221/evalopt-graph/blob/main/CHANGELOG.md)

## Maintenance and security

This repository is in maintenance: new features, integrations and root exports are not accepted.
Bug reports against `0.1.x` with a version and a minimal reproduction, corrections to the
documentation and migration map, and private security reports remain open. Start with
the [contribution guide](https://github.com/ajaysurya1221/evalopt-graph/blob/main/CONTRIBUTING.md) and
follow the [Code of Conduct](https://github.com/ajaysurya1221/evalopt-graph/blob/main/CODE_OF_CONDUCT.md).

Please report vulnerabilities privately using
[GitHub Security Advisories](https://github.com/ajaysurya1221/evalopt-graph/security/advisories/new), not a
public issue. See the [security policy](https://github.com/ajaysurya1221/evalopt-graph/blob/main/SECURITY.md) for supported versions and the bounded maintenance statement.

## Development

```bash
git clone https://github.com/ajaysurya1221/evalopt-graph.git
cd evalopt-graph
uv venv .venv
source .venv/bin/activate
uv pip install -e ".[dev]"

python -m compileall -q src tests scripts
python -m pytest -q
python -m ruff check src tests scripts
python -m ruff format --check src tests scripts
bash scripts/smoke_test.sh
uv build
```

Maintained by Ajay Surya Senthilrajan, with AI pair-programming recorded in commit trailers.
See the tests, design records and release evidence linked here.

## License

Released under the [MIT License](https://github.com/ajaysurya1221/evalopt-graph/blob/main/LICENSE).
Copyright © 2026 Ajay Surya Senthilrajan.
