Metadata-Version: 2.2
Name: gepa-dapo-grn
Version: 0.4.0
Summary: GEPA-aware DAPO reinforcement learning with optional Global Response Normalization
Author: InfiniteMalice
License: MIT
Project-URL: Homepage, https://github.com/InfiniteMalice/gepa-dapo-grn
Project-URL: Repository, https://github.com/InfiniteMalice/gepa-dapo-grn
Project-URL: Issues, https://github.com/InfiniteMalice/gepa-dapo-grn/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch<3.0,>=2.0
Provides-Extra: hf
Requires-Dist: transformers<5.0,>=4.40; extra == "hf"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: tomli>=1.1.0; python_version < "3.11" and extra == "dev"

# gepa-dapo-grn

`gepa-dapo-grn` is a standalone reinforcement learning engine for research workflows. It is
GEPA-shaped but GEPA-agnostic, providing DAPO and MaxRL-style optimization, curriculum tracking,
safety controls, verifier-first hooks, and optional Global Response Normalization (GRN).

## What this library is

- **Standalone RL engine** with a stable public API under `gepa_dapo_grn.*`.
- **GEPA-shaped but GEPA-agnostic**: feedback is structured reward/tag/verifier dictionaries.
- **Supports DAPO or MaxRL + curriculum + safety + GRN** with conservative defaults and GRN
  disabled by default (`GRNConfig.enabled=False`). At runtime,
  `SafetyController.adjust_grn_config(...)` (and internally `SafetyController._apply_grn_adjustment`)
  can auto-enable GRN when excess risk (`max(0, risk_score - risk_tolerance)`) exceeds `grn_enable_threshold`.
  It is recommended to use `risk_tolerance=0.0` (the default) for straightforward behavior. Using a
  nonzero tolerance can suppress triggering; for example, a `risk_score` of 0.6 with a `risk_tolerance`
  of 0.2 yields an excess risk of 0.4.

## Backends

- **DAPO**: general GEPA-shaped RL with mixed reward dimensions, curriculum, safety control,
  and optional GRN.
- **MaxRL**: verifier-heavy training for binary or near-binary correctness settings.

MaxRL is typically best when tasks have robust validators (for example code generation, exact
math checks, executable test suites, and other verifier-backed tasks). Non-binary
ethics/alignment training generally still benefits from DAPO-style hybrid control.

Minimal backend selection snippet:

```python
from gepa_dapo_grn import (
    DAPOConfig,
    MaxRLConfig,
    TrainerBackendConfig,
    make_trainer,
)

trainer = make_trainer(
    policy=policy,
    optimizer=optimizer,
    backend_config=TrainerBackendConfig(backend="maxrl"),
    maxrl_config=MaxRLConfig(enabled=True, num_samples=4),
)

# DAPO backend example:
dapo_trainer = make_trainer(
    policy=policy,
    optimizer=optimizer,
    backend_config=TrainerBackendConfig(backend="dapo"),
    dapo_config=DAPOConfig(),
)
```

## Practical guidance (v0.3.0)

- **Verifier-first**: use `VerifierResult` and `GEPAFeedback.verifier` for pass/fail, scores,
  confidence, coverage, and diagnostics.
- **Composition curriculum**: `CurriculumTracker` tracks saturation and composition depth; use
  `TaskComposer` (or `SimpleTextComposer`) to generate harder tasks as easier ones saturate.
- **Soft gating option**: set `DAPOConfig(use_soft_gating=True)` to smoothly down-weight ratio
  outliers instead of hard clipping.
- **Deception handling policy**: do not apply built-in deception penalties. Treat deception-like
  signals as tags/risk/controller inputs (abstention, calibration, verifier constraints).
- **GRN placement guidance**: GRN is off by default. Prefer enabling only named policy/value
  modules via `include_modules`/`exclude_modules`, and keep probe/interpretability modules
  unwrapped unless explicitly included.

## Optional Graph-Active-DAPO

The package also includes optional graph-native feedback and Active-GRPO-style adaptive references.
These modules can be used independently or combined through `GraphActiveDapoTrainer`.

- `gepa_dapo_grn.graph` scores public graph artifacts: claims, evidence links, mechanisms,
  assumptions, constraints, contradictions, and answer alignment.
- `gepa_dapo_grn.active_grpo` tracks per-prompt references, chooses imitate/mixed/reinforce modes,
  and promotes policy candidates only through strict external-verifier and safety gates.
- Hidden chain-of-thought is not stored or rewarded. The graph layer uses public artifacts only:
  graph JSON, public assumptions, claims, evidence, tests, answer summaries, and verifier outputs.
- Reference promotion is disabled by default and cannot rely on model self-score alone when external
  verification is required.

See `docs/graph_active_dapo.md` and the `examples/graph_*demo.py` files for minimal usage.

## Install

```bash
pip install gepa-dapo-grn
```

Optional extras:

- `gepa-dapo-grn[hf]` adds HuggingFace integration helpers.
- `gepa-dapo-grn[dev]` installs test and formatting tools.

## Minimal example (CPU-safe)

```python
from gepa_dapo_grn import DAPOTrainer, DAPOConfig, GEPAFeedback, RewardMixerConfig

fb = GEPAFeedback(
    rewards={"truth": 1.0, "helpfulness": 0.5},
    tags={"risk_score": 0.1},
    verifier={"verifier_pass": 1.0, "verifier_confidence": 0.9},
    meta={"task_id": "demo"},
    abstained=False,
)
```


## Building and installing a local wheel safely

If you build multiple versions locally, avoid `pip install dist/*.whl` because pip will try to
install all matching wheel files (which can include multiple versions of this same package and
fail with `ResolutionImpossible`).

Use this sequence instead:

```bash
rm -rf build dist *.egg-info
python3 -m pip install --upgrade build tomli
python3 -m build
python3 scripts/install_local_wheel.py --prune-other-versions
```

This follows pip's suggested fix to remove conflicting versions before install:
by default, `scripts/install_local_wheel.py` only locates and installs the
current version wheel and does not delete files in `dist/`. Cleanup is opt-in:
`--remove-version <version>` deletes wheel files for the specified version(s),
and `--prune-other-versions` removes other non-current wheel files so pip
receives exactly one path.


## Publishing to PyPI safely

A common cause of `InvalidDistribution` during `twine upload dist/*` is stale or non-package
artifacts left in `dist/`. Use a clean build and upload only the current version artifacts:

```bash
rm -rf build dist *.egg-info
python3 -m pip install --upgrade build twine tomli
# Build isolation uses pyproject build-system pins (setuptools<77) for twine compatibility.
python3 -m build
PROJECT_VERSION=$(python - <<'PY2'
from pathlib import Path
try:
    import tomllib as toml
except ImportError:
    import tomli as toml
import re
from packaging.version import Version, InvalidVersion

data = toml.loads(Path('pyproject.toml').read_text(encoding='utf-8'))
project = data.get('project', {})
version = project.get('version')
if isinstance(version, str) and version.strip():
    print(version.strip())
else:
    # Dynamic version fallback: read built artifact name from dist.
    # Expect files like gepa_dapo_grn-0.3.0-py3-none-any.whl
    wheel_names = list(Path('dist').glob('gepa_dapo_grn-*.whl'))
    if not wheel_names:
        raise SystemExit("No wheel found in dist for dynamic version resolution")
    
    def get_version(path):
        match = re.match(r"gepa_dapo_grn-([^-]+)-", path.name)
        if not match:
            raise SystemExit(f"Unable to parse version from wheel name: {path.name}")
        try:
            return Version(match.group(1))
        except InvalidVersion:
            raise SystemExit(f"Invalid semantic version in wheel name: {path.name}")

    newest_wheel = max(wheel_names, key=get_version)
    match = re.match(r"gepa_dapo_grn-([^-]+)-", newest_wheel.name)
    print(match.group(1))
PY2
)
python scripts/validate_dist_metadata.py \
  dist/gepa_dapo_grn-${PROJECT_VERSION}.tar.gz \
  dist/gepa_dapo_grn-${PROJECT_VERSION}-*.whl
twine check dist/gepa_dapo_grn-${PROJECT_VERSION}.tar.gz dist/gepa_dapo_grn-${PROJECT_VERSION}-*.whl
twine upload dist/gepa_dapo_grn-${PROJECT_VERSION}.tar.gz dist/gepa_dapo_grn-${PROJECT_VERSION}-*.whl
```

This avoids uploading unrelated files and ensures both `Name` and `Version` metadata come from the
freshly built distributions only.

If `twine check` reports supported metadata versions only up to `2.2` while your wheel has a newer
`Metadata-Version` (for example `2.4` from newer setuptools), upgrade your upload tooling first:

```bash
python -m pip install --upgrade twine pkginfo
```

## Public API

Public API is defined by `__init__.py` exports. Anything not exported there is considered
internal and may change without notice.

## Versioning policy

This project follows semantic versioning:

- `0.x.y` while interfaces are still evolving
- bump **minor** for interface changes
- bump **patch** for bugfixes only

See [CHANGELOG.md](CHANGELOG.md) for release notes.

## License

MIT (see [LICENSE](LICENSE)).
