Metadata-Version: 2.4
Name: auto-re-agent
Version: 0.2.1
Summary: Evidence-driven autonomous reverse-engineering agent with agentic Ghidra analysis, independent LLM verification, candidate validation, and parity checks
Project-URL: Homepage, https://github.com/dryxio/auto-re-agent
Project-URL: Repository, https://github.com/dryxio/auto-re-agent
Project-URL: Issues, https://github.com/dryxio/auto-re-agent/issues
Author: Dryxio
License-Expression: MIT
License-File: LICENSE
Keywords: ai-agent,autonomous-agent,binary-analysis,claude,codex,cybersecurity,decompilation,ghidra,llm,reverse-engineering,source-code-recovery,static-analysis
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Disassemblers
Requires-Python: >=3.10
Requires-Dist: anthropic>=0.40
Requires-Dist: openai>=1.0
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: mypy>=1.13; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.8; extra == 'dev'
Provides-Extra: ghidra-bridge
Requires-Dist: ghidra-ai-bridge>=0.2.0; extra == 'ghidra-bridge'
Provides-Extra: headless
Requires-Dist: ghidra-ai-bridge[headless]>=0.2.0; extra == 'headless'
Description-Content-Type: text/markdown

# auto-re-agent

[![PyPI](https://img.shields.io/pypi/v/auto-re-agent)](https://pypi.org/project/auto-re-agent/)
[![Python](https://img.shields.io/pypi/pyversions/auto-re-agent)](https://pypi.org/project/auto-re-agent/)
[![CI](https://github.com/Dryxio/auto-re-agent/actions/workflows/ci.yml/badge.svg)](https://github.com/Dryxio/auto-re-agent/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

`auto-re-agent` is an open-source AI reverse-engineering agent that uses Ghidra
and LLMs—including Claude, Codex, and OpenAI-compatible models—to reconstruct
and validate C/C++ functions from compiled binaries. It combines independent
reverser/checker models, agentic evidence gathering, candidate build and test
gates, structural verification, and parity analysis in one autonomous workflow.

Original pre-0.2 demo: [YouTube](https://youtu.be/zBQJYMKmwAs?si=emi1kDsJ81-2-tc3)

## What it does

```text
re-agent reverse --class CTrain
    │
    ├── Configuration (YAML + supported environment overrides + CLI flags)
    ├── Function selection (dependency-order | easiest-first | high-impact)
    ├── Source and binary context
    │   ├── decompile, xrefs, structs, enums, vtables, globals, and strings
    │   └── normalized high P-code, CFG, assembly, and nearby project source
    ├── Reverser → checker → fix loop (bounded rounds and investigations)
    ├── Conservative structural verifier
    ├── Candidate overlay
    │   └── configured build, test, and runtime gates
    ├── Candidate parity gate (GREEN | YELLOW | RED)
    └── Reports, per-round logs, session history, and knowledge graph
```

The tool generates candidate C/C++ implementations; it does not patch the
original source tree automatically. A successful reversal can require four
independent conditions:

1. the LLM checker returns `PASS`;
2. the objective verifier finds no strong structural mismatch;
3. candidate validation satisfies the configured acceptance policy;
4. parity is not blocked by the configured RED/YELLOW policy.

This is conservative verification, not a proof of semantic equivalence.

## Requirements

- Python 3.10+
- Git, for the current source installation
- Ghidra plus a configured
  [ghidra-ai-bridge](https://github.com/Dryxio/ghidra-ai-bridge)
- At least one LLM setup:
  - Claude API: `ANTHROPIC_API_KEY`
  - OpenAI-compatible API: `OPENAI_API_KEY`
  - Claude CLI: an authenticated local `claude` command
  - Codex CLI: an authenticated local `codex` command

## Installation

Install the agent and its Ghidra query bridge from PyPI:

```bash
python3 -m pip install --upgrade "auto-re-agent[ghidra-bridge]>=0.2.0"
```

For headless Ghidra exports, install the bridge with its PyGhidra extra:

```bash
python3 -m pip install --upgrade "auto-re-agent[headless]>=0.2.0"
```

To install the latest development versions directly from GitHub instead:

```bash
python3 -m pip install --upgrade \
  "ghidra-ai-bridge @ git+https://github.com/Dryxio/ghidra-ai-bridge.git@main" \
  "auto-re-agent @ git+https://github.com/Dryxio/auto-re-agent.git@main"
```

## Set up Ghidra evidence

Run these commands from the project you want to reverse:

```bash
# Create ghidra-bridge.yaml, then edit its Ghidra project/program paths
ghidra-bridge init

# Requires the bridge headless extra and a local Ghidra installation
ghidra-bridge export all

# Optional but recommended when reversed source/hook patterns are available
ghidra-bridge build-map

# Confirm that exports and configuration are visible
ghidra-bridge info
```

See the [bridge documentation](https://github.com/Dryxio/ghidra-ai-bridge)
for its Ghidra, export, and source-map configuration.

## Quick start

Create a configuration in the target project:

```bash
# Recommended portable default
re-agent init --profile generic-cpp

# Other available profiles
# re-agent init --profile windows-x64
# re-agent init --profile gta-reversed
# re-agent init --profile openrct2
```

Running `re-agent init` without `--profile` preserves the original
GTA-reversed defaults. Prefer an explicit profile for new projects.

Then edit `re-agent.yaml`. At minimum, select an LLM, point the backend at the
installed bridge executable, set the source paths, and configure validation.

```yaml
llm:
  provider: claude-cli
  model: sonnet

# Optional: use a different provider/model for checking.
agents:
  checker:
    provider: codex
    model: gpt-5.4

backend:
  type: ghidra-bridge
  cli_path: ghidra-bridge

project_profile:
  name: generic-cpp
  language_standard: C++20
  source_root: src
  hooks_csv: null

orchestrator:
  max_review_rounds: 4
  investigation_enabled: true
  max_investigations: 8
  selection_strategy: dependency-order
  max_attempts_per_function: 3

validation:
  enabled: true
  copy_project: true
  project_root: .
  build_commands:
    - cmake -S . -B build
    - cmake --build build
  test_commands:
    - ctest --test-dir build --output-on-failure
  require_build: true
  require_tests: true
  require_verified: true
  # This explicitly attests that the project-owned shell commands above are
  # meaningful validation gates. Leave false for untrusted commands.
  trust_configured_commands: true
  keep_project_copy: false
  parity_fail_on_red: true
  parity_fail_on_yellow: false
```

Validation is deliberately strict: with the generated defaults, no configured
commands produce `UNKNOWN`, and `require_verified: true` rejects that result.
For exploration without build validation, explicitly set
`validation.enabled: false`; such results are not build-verified.

Start with one function before launching a class run:

```bash
re-agent reverse --address 0x401000
re-agent reverse --class CTrain --max-functions 10
re-agent status
```

## LLM providers

### Claude API

```yaml
llm:
  provider: claude
  model: claude-sonnet-4-5-20250929
```

Set `ANTHROPIC_API_KEY` or `RE_AGENT_LLM_API_KEY`.

### Claude CLI

Authenticate the local Claude Code CLI first, then configure:

```yaml
llm:
  provider: claude-cli
  model: sonnet
  cli_path: claude
  effort: high
  max_budget_usd: 1.0
```

Claude CLI supports real session resume and reports usage/cost metadata. A
stale CLI login can still require re-authentication even when its auth-status
command reports a session.

### OpenAI-compatible APIs

```yaml
llm:
  provider: openai # or openai-compat
  model: your-model
  base_url: https://your-endpoint.example/v1 # optional
```

Set `OPENAI_API_KEY` or `RE_AGENT_LLM_API_KEY`.

### Codex CLI

```yaml
llm:
  provider: codex
  model: gpt-5.4
```

Codex uses the authenticated local `codex exec` command. CLI-provider
`max_tokens` values are planning allowances, not hard output limits.

Omit `agents.reverser` or `agents.checker` to reuse the top-level `llm`
configuration for that role. A role block is a complete role configuration,
not a field-by-field merge with `llm`.

## Evidence and investigation

When supported by the backend, the reverser preloads a bounded evidence bundle
and can request additional read-only operations:

- `decompile`, `xrefs_from`, and `xrefs_to`
- `struct` and `enum`
- `vtable`, `global`, and `strings`
- `context`, normalized `pcode`, and `cfg`

Evidence bundle data is also ingested into
`reports/re-agent/knowledge-graph.json`, connecting functions, calls, globals,
and strings. Unsupported bridge capabilities degrade gracefully.

## Candidate validation

Generated code is written to an overlay. With `copy_project: true`, the project
is copied to a temporary directory, the candidate replaces the matching body
there, and commands run from that copy. `.git`, `.venv`, `build`, `reports`, and
Python cache files are not copied. Temporary project copies are deleted unless
`keep_project_copy: true`.

Commands may use:

- `{candidate_file}`, `{overlay_root}`, and `{source_file}` placeholders;
- `RE_AGENT_CANDIDATE_FILE`, `RE_AGENT_OVERLAY_ROOT`, and
  `RE_AGENT_SOURCE_FILE` environment variables.

Configured build/test/runtime commands are arbitrary project-owned shell
commands. The agent cannot prove from their text that they actually validate a
candidate, so they only become acceptance evidence when
`trust_configured_commands: true` is set explicitly.

If multiple C++ definitions match an overloaded method and the source cannot be
disambiguated, the overlay is rejected instead of replacing an arbitrary body.

## Verification and parity

The objective verifier runs on each review round. It compares generated code
with available decompile, assembly, CFG, and normalized high P-code evidence.
It returns `FAIL` only for strong mismatches; insufficient evidence returns
`UNKNOWN`.

The reversal pipeline runs the 11 built-in heuristic parity signals against the
generated candidate body. RED is blocking by default; YELLOW can be made
blocking with `validation.parity_fail_on_yellow`.

The standalone command is different: `re-agent parity` analyzes functions in
the existing source tree. It also supports semantic-rule files and manual check
overrides. Its process exit code remains zero on RED unless `--strict-exit` is
used.

The 11 built-in signals are:

| Signal | Level | Description |
|---|---|---|
| Missing source | RED | No source body was found |
| Stub markers | RED | Source contains a configured stub marker |
| Trivial stub | RED | Small plugin-call-heavy body with no control flow |
| Large ASM, tiny source | RED | Large disassembly with a very small source body |
| Plugin-call heavy | YELLOW | Plugin calls dominate the source body |
| Short body | YELLOW | Body has fewer than six lines |
| Low call count | YELLOW | Decompiled callees greatly exceed source calls |
| FP sensitivity | YELLOW | Assembly has FP-sensitive operations but source has no math tokens |
| Call-count mismatch | YELLOW | Source and assembly call counts differ beyond the configured threshold |
| NaN logic | YELLOW | Decompile indicates NaN-sensitive behavior missing from source |
| Inline wrapper | INFO | Source forwards to an internal implementation |

The signal set is fixed in `0.2.0`; configuration exposes selected thresholds,
inline-wrapper behavior, semantic rules, and manual overrides rather than an
individual toggle for every signal.

## CLI reference

Global options must precede the subcommand, for example
`re-agent --config custom.yaml status`.

| Command | Purpose |
|---|---|
| `re-agent init --profile generic-cpp` | Create `re-agent.yaml` from a profile |
| `re-agent reverse --address ADDR` | Reverse one function |
| `re-agent reverse --class CLASS --max-functions N` | Reverse a bounded class batch |
| `re-agent reverse --class CLASS --dry-run` | Show a target plan without LLM calls |
| `re-agent reverse ... --max-rounds N --skip-parity` | Override loop/parity behavior |
| `re-agent parity --address ADDR --strict-exit` | Analyze an existing source function |
| `re-agent parity --filter REGEX --limit N --output report.json` | Filter and export parity results |
| `re-agent parity ... --skip-ghidra` | Run source-only parity signals |
| `re-agent status --class CLASS --format text` | Show session progress |
| `re-agent estimate --address ADDR` | Estimate one function |
| `re-agent estimate --class CLASS --limit N` | Estimate a class batch |

Use `re-agent <command> --help` for the exact option list.

## Configuration precedence

The effective order is CLI runtime overrides, supported environment variables,
`re-agent.yaml`, then dataclass defaults. The currently supported environment
variables are:

- `RE_AGENT_LLM_PROVIDER`
- `RE_AGENT_LLM_API_KEY`
- `RE_AGENT_LLM_MODEL`
- `RE_AGENT_LLM_BASE_URL`
- `RE_AGENT_BACKEND_CLI_PATH`
- `RE_AGENT_BACKEND_TIMEOUT`

Role-specific `agents.*` configuration, validation, project profiles, parity,
and output paths should be configured in YAML.

See [docs/configuration.md](docs/configuration.md) for the complete schema.

## Profiles

- `generic-cpp`: portable C/C++ defaults
- `windows-x64`: Microsoft x64-oriented prompt rules
- `gta-reversed`: GTA-reversed hooks, stubs, source paths, and project rules
- `openrct2`: OpenRCT2-oriented hook/stub patterns

Profiles initialize project configuration; they do not replace bridge exports
or project-specific validation commands.

## Outputs

Default artifacts include:

- `reports/re-agent/code/`: final generated code per function
- `reports/re-agent/logs/`: per-round reverser/checker prompts, responses, and provider metadata
- `reports/re-agent/candidates/`: non-isolated candidate overlays
- `reports/re-agent/knowledge-graph.json`: persistent evidence graph
- `re-agent-progress.json`: current per-function state plus run history

The session file is atomically rewritten on save. Its `functions` map stores the
latest state per address, while its `runs` list preserves recorded attempts.

## How it compares

| Approach | Primary use | Evidence and validation | Workflow |
|---|---|---|---|
| Traditional decompiler | Translate machine code into analyst-readable pseudocode | Decompiler analysis; correctness is assessed manually | Function-by-function analysis |
| Interactive Ghidra AI or MCP assistant | Let an analyst ask questions and request Ghidra operations | Depends on the analyst, prompts, and connected tools | Human-directed conversation |
| `auto-re-agent` | Generate and validate candidate C/C++ implementations | Ghidra evidence, independent checker, structural checks, configured build/tests, and parity signals | Bounded autonomous reverser/checker pipeline with persistent reports |

`auto-re-agent` complements Ghidra rather than replacing it: Ghidra supplies
the program analysis, while the agent orchestrates evidence collection,
implementation, review, validation, and reporting. It is designed for
repeatable project-scale workflows, not just one-off decompiler chat.

## Frequently asked questions

### Is auto-re-agent a decompiler?

Not in the traditional sense. Ghidra performs the disassembly, decompilation,
and program analysis. `auto-re-agent` uses that evidence plus project source
context and LLMs to produce and validate candidate C/C++ implementations.

### Does it require Ghidra?

The full binary-backed reversal workflow currently uses Ghidra through
`ghidra-ai-bridge`. Existing source can be checked with source-only parity via
`re-agent parity --skip-ghidra`, but that mode has less evidence.

### Which LLM providers are supported?

Claude API, Claude CLI, OpenAI-compatible APIs, and Codex CLI are supported.
The reverser and checker can use different providers or models.

### Does it modify the original source tree?

No. Generated implementations are written to reports and candidate overlays.
When isolated validation is enabled, builds and tests run in a temporary copy
of the project.

### Can it prove that generated source is equivalent to the binary?

No. The checker, structural verifier, configured build/test gates, and parity
signals provide conservative evidence, not a formal proof of semantic or
binary equivalence.

### What binaries and projects can it analyze?

It can work with programs that Ghidra can import and that the bridge can export.
Useful reconstruction also depends on project-specific source context, types,
symbols, validation commands, and the evidence available in the target binary.

### How are LLM cost and run length controlled?

Review rounds, investigations, and attempts per function are bounded in the
configuration. Provider logs record available usage and cost metadata; actual
cost depends on the selected models, evidence volume, and target complexity.

## Safety and limitations

- re-agent does not commit or push generated code;
- candidate generation does not overwrite the original source tree;
- review rounds, evidence actions, and per-function attempts are bounded;
- prompt/response logs are written per review round, not for every internal
  evidence-loop call;
- configured validation commands execute through `/bin/sh` and should only be
  trusted when they are controlled by the project owner;
- structural and parity checks catch useful mismatches but do not prove binary
  equivalence;
- real Ghidra/PyGhidra integration depends on the local Ghidra project and has
  to be tested in that environment.

## Why ghidra-ai-bridge stays separate

`ghidra-ai-bridge` remains an independent analysis package with a versioned
JSON/CLI evidence surface. auto-re-agent consumes it through a capability-based
backend, leaving room for future IDA, Binary Ninja, or other backends.

## Development

```bash
git clone https://github.com/Dryxio/auto-re-agent.git
git clone https://github.com/Dryxio/ghidra-ai-bridge.git
cd auto-re-agent

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -e "../ghidra-ai-bridge[headless]"
python3 -m pip install -e ".[dev]"

pytest -q
ruff check src tests
mypy src
```

## License

MIT
