Metadata-Version: 2.4
Name: pymimir-rl
Version: 0.3.0b3
Summary: Reinforcement Learning (RL) package for Mimir.
Author-email: Simon Stahlberg <simon.stahlberg@gmail.com>
License-Expression: GPL-3.0-or-later
Project-URL: Homepage, https://github.com/simon-stahlberg/mimir-rl
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.6.0
Requires-Dist: pymimir==0.14.0b5
Requires-Dist: networkx>=3.5
Provides-Extra: dev
Requires-Dist: pymimir-rgnn==0.3.0b4; extra == "dev"
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Dynamic: license-file

# Mimir-RL

Mimir-RL is a Python library that implements RL algorithms using PyTorch and PyTorch RL that are tightly integrated with Mimir.

## Dead-end detection

Pass an optional state/goal detector callable to a trajectory sampler:

```python
import pymimir as mm
from pymimir_rl import BoltzmannTrajectorySampler, CachedDeadEndDetector

detector = CachedDeadEndDetector(mm.H2DeadEndDetector)
sampler = BoltzmannTrajectorySampler(
    model, reward_function, temperature=0.5, dead_end_detector=detector,
)
```

For h², construct problems with `generator="grounded"`. The cache creates one
native detector per problem. `Trajectory` calls it after sampling, in state order,
and propagates each proof forward for the same goal. Existing reward-function
proofs and actionless non-goal states also provide dead-end evidence.

Detected states with applicable actions do not terminate or prune rollouts.
`Transition.successor_is_dead_end` is separate from `is_terminal`, and ordinary
rewards are preserved. Optimizers assign fixed dead-end targets without
bootstrapping from those successors. Hindsight cloning recomputes labels for its
new goal using the same detector cache.

`OffPolicyAlgorithm` accepts `dead_end_replay_buffer` and `hindsight_replay_buffer`.
The former receives full trajectories that contain a proven dead state, including
exploration after that state. All sampled trajectories remain available for
hindsight refinement. An unproven horizon cutoff alone does not qualify for the
dead-end buffer.
