Metadata-Version: 2.4
Name: gradecheck
Version: 0.1.0
Summary: Attack-test RL environment graders for reward hacking — a pentest for reward functions.
Project-URL: Homepage, https://github.com/PoorvaJ-WW/gradecheck
Project-URL: Repository, https://github.com/PoorvaJ-WW/gradecheck
Author: Poorva Jain
License-Expression: Apache-2.0
Keywords: evaluation,grader,qc,reward-function,reward-hacking,rl,rlvr
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.11
Provides-Extra: verifiers
Requires-Dist: verifiers>=0.2; extra == 'verifiers'
Description-Content-Type: text/markdown

# gradecheck

**Attack-test your RL environment's grader before an RL run does it for you.**

RL environments reward whatever the grader accepts — and models optimize the
grader, not the task. If the grader has a loophole (delete the failing tests,
copy the expected outputs, submit nothing and exit clean), training finds it and
you ship a model that learned to cheat. Today those loopholes are caught by a
human reading grader code and imagining exploits.

`gradecheck` makes the *known* exploits a checklist that runs automatically. You
point it at an environment; it plays the laziest, most dishonest agent possible —
running a battery of canonical cheap strategies against your grader — and reports
which ones were wrongly rewarded:

```
$ gradecheck run ./my-env

  ⚠ FOOLED   test-deletion  Grader awards full reward (1.000) after deleting 1 test file(s).
  ⚠ FOOLED   test-deletion  Grader awards full reward (1.000) after emptying 1 test file(s).
  △ WARN     answer-leak    1 answer-shaped file(s) reachable from the submission tree.
  ✓ RESISTED noop           Starting tree scores 0.000 (not solved) — correct.
  ✓ RESISTED empty-source   Empty submission scores 0.000 <= baseline 0.000 — correct.

  VULNERABLE — grader fooled by 2 attack(s)
```

Exit code is `1` when any attack fools the grader, so it drops straight into CI.

It does **not** catch every possible hack — nothing can. It clears out the known,
mechanical loopholes so human review spends its imagination on the novel ones.

## Install

```
pip install gradecheck
```

Zero dependencies. Python ≥ 3.11. (The `verifiers` integration below adds one
optional dependency and currently wants Python 3.11–3.12.)

## Setup

Drop a `gradecheck.toml` next to your environment:

```toml
[env]
name = "my-task"
workspace = "task"                 # the tree the agent edits and submits
source_globs = ["src/**/*.py"]     # files a real fix would touch

[grade]
cmd = "python3 verification/grade.py --task-dir {workspace}"
timeout = 1800
pass_marker = "RESULT: PASS"       # exit 0 + this marker == solved
score = [                          # optional: partial-credit scoreboard lines
  { regex = 'F2P:\s*(\d+)\s*/\s*(\d+)' },
  { regex = 'P2P:\s*(\d+)\s*/\s*(\d+)' },
]
```

`gradecheck` copies `workspace` fresh for every attack, mutates the copy the way
a cheating agent would, runs your grade command against it, and compares the
reward to the honest baseline. Your environment is never modified.

## Attacks

| id | what it simulates |
|---|---|
| `noop` | submit the starting tree unchanged — it must not already pass |
| `empty-source` | blank out the source files — absence must never score higher |
| `test-deletion` | delete the visible tests, then separately empty them — the pass signal must not depend on tests the agent can rewrite |
| `answer-leak` | (static) scan the submission tree for reachable gold/expected/answer artifacts an agent could copy |
| `hollow-implementation` | read the grader's source (`[grade].source`), extract the string literals/paths it checks for, write a fake artifact containing them, and see if the grader passes without the task being done |

Verdicts: **FOOLED** (grader gave undeserved reward — fix it), **WARN** (exploit
surface a human should confirm), **RESISTED** (correct behavior).

```
gradecheck attacks              # list attacks
gradecheck run . --json         # machine-readable report
gradecheck run . --attacks noop,test-deletion
```

## Prime Intellect `verifiers` environments

Verifiers environments grade a model *completion* rather than a file tree — the
author's reward functions live under `env.rubric`. gradecheck attacks them
directly: it loads the environment, samples real task examples, and scores
degenerate completions against the reward functions. No rollout, no Docker, no
LLM, no API key.

```
pip install "gradecheck[verifiers]"
prime env install some-env            # or pip install the env package
gradecheck verifiers some-env --sample 20
```

```
  ⚠ FOOLED   flattery       Judge-flattery / correctness assertion: accepted on 3/3 sampled example(s).
  ⚠ FOOLED   prompt-echo    Prompt echoed back: accepted on 3/3 sampled example(s).
  ✓ RESISTED empty-completion Empty completion: rejected on all 3 sampled example(s).
  ✓ RESISTED garbage        Nonsense completion: rejected on all 3 sampled example(s).
  ✓ RESISTED answer-leak    Gold answer not present in the agent's observation.
  · INFO     baseline       Gold answer scores as solved on 3/3 sampled example(s).
```

Completion attacks: `empty-completion`, `prompt-echo`, `garbage`,
`flattery` (bare assertions of correctness — catches keyword/LLM-judge rewards),
and `answer-leak` (checks whether the gold answer is visible in the agent's own
observation, not merely accessible to the reward function). A reward function
that accepts any of these is scoring surface form, not correctness.

## Writing your own attacks

Attacks are small classes with a registry — subclass `gradecheck.attacks.base.Attack`,
implement `run(ctx)`, and call `gradecheck.attacks.register(MyAttack)`. `ctx` gives
you `materialize()` (fresh workspace copy), `grade(ws)`, and the honest `baseline`.

## Safety note

`gradecheck` deliberately submits adversarial trees to *your own* grader, on your
machine, using the grade command *you* configure. It executes nothing an agent
couldn't already make your grader execute — but as with any grading run, do it
inside the same sandbox/container your environment normally grades in.

## License

Apache-2.0
