Metadata-Version: 2.4
Name: goalie
Version: 0.1.0
Summary: Verification harness for AI-generated code
Project-URL: Repository, https://github.com/MatteoPossamai/goalie
Project-URL: Issues, https://github.com/MatteoPossamai/goalie/issues
Author: Matteo Possamai
License-Expression: MIT
License-File: LICENSE
Keywords: ai,hypothesis,property-based,refactoring,testing,verification
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Requires-Dist: hypothesis>=6.100
Description-Content-Type: text/markdown

# goalie

Verification harness for AI-generated code. Checks that a diff actually does
what it should, using oracles independent of whoever (or whatever) wrote it —
and returns machine-readable verdicts an agent loop can act on.

## Commands

```bash
goalie diff-test     # old vs new implementation on generated inputs
goalie properties    # generate + run property-based tests  [planned]
goalie mutate        # property tests by injecting bugs     [planned]
goalie bench         # performance regression check         [planned]
```

### diff-test

For refactors, the old implementation is a free oracle. `goalie` extracts changed
functions from the git diff, generates inputs with Hypothesis, and runs both versions:

```json
{
  "target": "core/rolling.py::rolling_max",
  "verdict": "FAIL",
  "examples": 2000,
  "seed": 42,
  "counterexample": { "kwargs": { "xs": "[1, 2, 2]", "w": "2" }, "old": "2", "new": "1" }
}
```

Deterministic, reproducible from the seed, no LLM involved. Add one line to your
AGENTS.md / CLAUDE.md and your agent has to prove its refactor preserved
behavior before calling it done.

### properties [planned]

An LLM proposes Hypothesis invariants for a function; they're written to `.goalie/`
as plain code you review and commit. Generation uses a model once — execution
is deterministic forever.

### mutate [planned]

Trust check for the property suite: inject small bugs (<= → <, max → min) and re-run.
Properties that don't catch planted bugs won't catch real ones.
Outputs a mutation score.

### bench [planned]

pyperf old vs new with a regression threshold.
Catches the O(n²) that passes every test.

## Roadmap

v0 — diff-test end-to-end on pure-ish typed functions
v0.1 — pipx install goalie, CI exit codes, kwargs/exceptions handling
v1 — properties + mutate
v1.x — MCP server mode, bench, stateful testing

Out of scope: sandboxing, non-Python, greenfield code with nothing to diff against.

## Status

diff-test v0 implemented. Design spec:
[docs/superpowers/specs/2026-07-24-diff-test-design.md](docs/superpowers/specs/2026-07-24-diff-test-design.md)

### Quickstart

```bash
uv sync
uv run goalie diff-test
```

Report written to `.goalie/latest.json` and `.goalie/reports/`. To waive an
intentional behavior change:

```bash
uv run goalie waive path/to/file.py::function_name --reason "<why>"
```

### Running tests

Agent-verifiable suite (green):

```bash
uv run pytest --ignore=tests/human -q
```

Gated suite — red until the maintainer implements the human-owned modules
(`src/goalie/discovery.py`, `src/goalie/orchestrator.py`):

```bash
uv run pytest tests/human -v
```
