Metadata-Version: 2.4
Name: morality-gym
Version: 0.1.0
Summary: Gymnasium-compatible trolley-problem environments for studying reinforcement-learning agents under moral constraints.
Author: Simon Rosen
License: MIT
Project-URL: Homepage, https://github.com/SimonRosen173/morality-gym-tabular
Project-URL: Repository, https://github.com/SimonRosen173/morality-gym-tabular
Keywords: reinforcement-learning,environment,gymnasium,safe-rl,moral-ai
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: numpy
Requires-Dist: matplotlib
Requires-Dist: gymnasium

# Morality Gym Tabular

Morality Gym Tabular provides Gymnasium-compatible trolley-problem environments for studying how reinforcement-learning agents behave under moral constraints.

The repository has two layers:

- `morality_gym/`: the environment package, morality chains, cost function utilities, and lightweight wrappers.
- `baselines/` and `omnisafe/`: experiment code used for the paper benchmarks. These are intended for repository users, not as the minimal pip-facing API.

## Installation

For development from this repository:

```bash
git clone https://github.com/SimonRosen173/morality-gym-tabular.git
cd morality-gym-tabular
pip install -e .
```

The lightweight environment path depends on `numpy`, `matplotlib`, and `gymnasium`. The experiment stack also uses additional packages such as `torch`, `stable-baselines3`, `omnisafe`, `pandas`, `tqdm`, and cluster tooling where relevant.

## Basic Usage

```python
from morality_gym import make

env, morality_chain = make(
    env_id="SwitchStandard-HumanA-v1",
    morality_chain_id="Utility",
)

obs, info = env.reset(seed=42)

done = False
while not done:
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    done = terminated or truncated

env.close()
```

Environment IDs use:

```text
{scenario_id}-{variant_id}-v1
```

Examples include `SwitchStandard-HumanA-v1`, `PushStandard-HumanA-v1`, `Switch5-Human-v1`, and `PushOrSwitch-Human-v1`.

## Safe-RL Example

The standalone example in [examples/train_safe_rl_agent.py](examples/train_safe_rl_agent.py) shows how to:

1. create an environment and morality chain,
2. wrap the environment so `info["cost"]` is emitted at each step,
3. train a small constrained Q-learning agent with a Lagrangian cost penalty.

Run a short smoke test:

```bash
python examples/train_safe_rl_agent.py \
  --episodes 10 \
  --eval-episodes 3 \
  --max-episode-steps 25 \
  --log-every 5
```

Run a longer example:

```bash
python examples/train_safe_rl_agent.py --episodes 500 --eval-episodes 50
```

This script is a readable demonstration of the cost-wrapper pattern. It is not the paper benchmark implementation.

## Experiments

Paper-style experiments are configured and run through the benchmarker under [baselines/](baselines/README.md). In short:

```bash
python baselines/cli.py --create exp_p9xe.json
python baselines/cli.py --exec-run p9xe/run_0.json
```

See [baselines/README.md](baselines/README.md) and [baselines/benchmarker/README.md](baselines/benchmarker/README.md) for the experiment configuration format, local execution, SLURM execution, logs, and result consolidation notes.

## Documentation

| Resource | Description |
| --- | --- |
| [morality_gym/README.md](morality_gym/README.md) | Supported environment ID syntax and scenarios |
| [examples/](examples/) | Small scripts for interacting with and evaluating environments |
| [baselines/README.md](baselines/README.md) | Baseline learner and benchmarker overview |
| [baselines/benchmarker/README.md](baselines/benchmarker/README.md) | Experiment configuration and execution details |

## Citation

If you use this framework in your research, please cite:

```bibtex
@misc{rosen2024moralitygym,
  author = {Rosen, Simon and Singh, Siddarth and Robertson, Helen Sarah and Gelo, Ebenezer and
            Suder, Ibrahim and Nangue Tasse, Geraud and Williams, Victoria and
            James, Steven and Rosman, Benjamin},
  title = {Morality Gym Tabular: A Framework for Moral Dilemmas in Reinforcement Learning},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub Repository},
  howpublished = {\url{https://github.com/SimonRosen173/morality-gym-tabular}}
}
```

## Contributing

Contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md).

## License

This project is licensed under the terms of [LICENSE](LICENSE).
