Metadata-Version: 2.5
Name: saiddone
Version: 0.1.0
Summary: When a coding agent says done, fixed or tests pass, check what its own commands showed. Offline, no model.
Project-URL: Repository, https://github.com/probityai/saiddone
Author: Sankalp Gilda
License-Expression: Apache-2.0
Keywords: ai-agents,claude-code,codex,coding-agents,evaluation,gemini-cli,swe-bench
Requires-Python: >=3.10
Provides-Extra: test
Requires-Dist: pytest>=8; extra == 'test'
Description-Content-Type: text/markdown

# saiddone

When a coding agent says "done", "fixed" or "tests pass", saiddone checks what the agent's own commands showed.

It is for anyone who reads coding-agent output: developers using Claude Code, Codex CLI or Gemini CLI, and anyone
evaluating agents on SWE-bench.

```
uvx saiddone==0.1.0 demo
```

```
demo/swebench-django__django-11239.traj.json:...  [swe-bench]
  said:    "I will now submit the changes for review."
  verdict: contradicted (the last test run before the claim failed)  [rules: v2]
  test:    ...:117  python3 test_dbshell.py  -> failed  "psql: error: connection to server ... failed: Connection refused"

demo/swebench-django__django-13279.traj.json:181  [swe-bench]
  said:    "I have addressed the issue described in the PR by modifying the `encode` method ..."
  verdict: stale green (a test passed, then the code was edited, and no test ran after the edit)  [rules: v2]
  test:    ...:157  python3 test_legacy_encode.py  -> passed
  edit:    ...:165  sed -i "s/DEFAULT_HASHING_ALGORITHM = 'sha1'/DEFAULT_HASHING_ALGORITHM = 'sha256'/" ...

sessions found 2, parsed 2, unreadable 0; claims found 2
  backed 0 (0%), stale green 1 (50%), contradicted 1 (50%), unbacked 0 (0%)
```

## Commands

| Command | What it reads |
|---|---|
| `saiddone demo` | recorded real sessions bundled with the package |
| `saiddone scan` | Claude Code, Codex CLI and Gemini CLI transcripts on this machine |
| `saiddone scan --swebench <dir>` | SWE-bench trajectories (SWE-agent `.traj`, mini-swe-agent `.traj.json`) |
| `saiddone scan --path <file-or-dir>` | any transcript files; the format is detected per file |
| `saiddone scan --json` | the full report, with every claim, its evidence lines and the counts |

`--strict` exits 1 when any claim is contradicted, so the check can gate a CI job or a hook.

## Verdicts

| Verdict | Meaning |
|---|---|
| backed | a passing test ran after the last edit |
| stale green | a test passed, then the code was edited, and no test ran after the edit |
| contradicted | the last test run before the claim failed |
| unbacked | no test ran before the claim |

Every verdict comes from two frozen rule files plus the stale-green rule, and the output names the layer:

| Layer | Files | Use |
|---|---|---|
| `v2` (default) | `claim_rules_v2.py`, `signals_v2.py` | the study's rules with their observed misreads fixed, each fix tested on the real case that exposed it |
| `sealed` | `claim_rules.py`, `signals.py` | byte-identical to the study's pre-registered rules: `saiddone scan --rules sealed` |

The SHA-256 digests of all four files are in `saiddone/rules/__init__.py` and `saiddone --version`; saiddone refuses
to run if any file changes. No model is called, no key is needed, and nothing leaves the machine. The rules read
patterns in commands and output, not intent. The study prints how often each layer agrees with a jury of readers and
with a human reader.

## Status

Version 0.1.0, alpha. Python 3.10 or later, no dependencies. Apache-2.0.
