Watch it block a bad AI change

01 The pull request

A data agent learns to answer free-form questions

On main, analysis-agent answers questions about a CSV by letting the model pick one named aggregation from an allowlist. Safe, boring, capped. Then a PR lands: “support natural-language queries.” It asks the model for a pandas expression and evaluates it against the dataframe. Every line looks reasonable in review — and a prompt-injected cell in the CSV now reaches eval().

    op = resp.choices[0].message.content.strip()
    if op not in ALLOWED:            # validate against an allowlist
        raise ValueError(op)
    return getattr(df, op)(numeric_only=True)
    expr = resp.choices[0].message.content
    return eval(expr, {"df": df})   # model output executed directly

02 The gate

One command, scoped to the diff

In the PR’s CI job — or locally before you push — one line. It compares the branch against its merge-base and rules on what this change introduced, never inherited debt.

03 The verdict

Clean on main → blocked on the PR

Same agent, two states. The baseline promotes. The change that added the eval flips it to a block — and names the exact line and why.

04 Why you can trust the red

A gate teams keep, not one they mute

• Net-new only. It blocks what this PR introduced and never fails you on debt you inherited — so the red always means “you just added this.”
• Confirmed, not guessed. The HIGH is graded confirmed because we can point at where the value came from: it was assigned from client.chat.completions.create(…) on line 17 and reaches eval() on line 25. Open those two lines and check us — every HIGH carries that chain.
• A variable’s name is never evidence. Scan the same service and cache.py has pickle.loads(payload) — a genuinely dangerous sink. A name-matching scanner calls that confirmed remote code execution because the variable is called payload. We report it as a MEDIUM that asks you to confirm the source, because it’s a bare parameter whose caller this file can’t see. In AutoGPT that same variable held the cache’s own HMAC-signed bytes — asserting a vulnerability there is how a report gets dismissed by a maintainer who knows their code.
  `payload` → pickle.loads() (L26) — origin unknown
• It’s just an exit code. 0 promote · 10 hold · 1 block. Drops into any CI without a dashboard, an account, or your code leaving the runner.

05 Run it yourself

The whole example is in the repo

Nothing here is staged. The agent versions and the script that builds the git scenario and runs the gate live under examples/demo-code-risk/. Reproduce the output above in about thirty seconds:

And we don’t rely on you to catch us. --check asserts the verdict, the score delta, the line numbers and the provenance chain quoted on this page, and CI runs it on every push. If the engine’s output ever drifts from what you read here, the build fails — so this page can’t quietly become a mockup.