Metadata-Version: 2.4
Name: comrad-bench
Version: 1.0.0
Summary: MARL benchmark suite in ViZDoom
Author-email: "Khoi H.B. Nguyen" <baokhoi136@gmail.com>, Tristan Tomilin <tristan.tomilin@hotmail.com>, Dimitar Zhekov <mitko.zh@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/TTomilin/COMRAD
Project-URL: Repository, https://github.com/TTomilin/COMRAD
Keywords: marl,vizdoom,reinforcement-learning,multi-agent,benchmark
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Games/Entertainment :: First Person Shooters
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy<2.0,>=1.26.4
Requires-Dist: gymnasium[classic-control]<1.0,>=0.27
Requires-Dist: pettingzoo<1.25.0
Requires-Dist: opencv-python>=4.11.0
Requires-Dist: pygame>=2.6.1
Requires-Dist: pre-commit>=4.5.1
Requires-Dist: pynput>=1.8.1
Requires-Dist: wandb[media]==0.22.1
Requires-Dist: torch>=2.10.0
Requires-Dist: torchrl>=0.11.0
Requires-Dist: torchvision>=0.25.0
Requires-Dist: torchaudio>=2.10.0
Requires-Dist: psutil>=7.2.2
Requires-Dist: pyglet>=2.1.12
Requires-Dist: tensorboard>=2.20.0
Requires-Dist: tensorboardx>=2.6.4
Requires-Dist: threadpoolctl>=3.6.0
Requires-Dist: colorlog>=6.10.1
Requires-Dist: signal-slot-mp<2.0,>=1.0.3
Requires-Dist: filelock>=3.20.3
Requires-Dist: huggingface-hub>=0.10.0
Requires-Dist: pandas>=2.3.3
Requires-Dist: onnx>=1.19.0
Requires-Dist: onnxruntime>=1.23.2
Requires-Dist: matplotlib>=3.10.8
Requires-Dist: seaborn>=0.13.2
Requires-Dist: scipy>=1.17.0
Requires-Dist: vizdoom>=1.3.0
Provides-Extra: sample-factory
Requires-Dist: torch>=2.9.0; extra == "sample-factory"
Requires-Dist: wandb>=0.22.1; extra == "sample-factory"
Requires-Dist: tensorboard>=2.20.0; extra == "sample-factory"
Requires-Dist: colorlog>=6.10.1; extra == "sample-factory"
Requires-Dist: signal-slot-mp>=1.0.5; extra == "sample-factory"
Requires-Dist: faster-fifo>=1.5.2; extra == "sample-factory"
Requires-Dist: filelock>=3.19.1; extra == "sample-factory"
Requires-Dist: psutil>=7.1.0; extra == "sample-factory"
Requires-Dist: huggingface_hub>=0.36.0; extra == "sample-factory"
Requires-Dist: scipy>=1.16.3; extra == "sample-factory"
Provides-Extra: visual
Requires-Dist: omgifol>=0.5.1; extra == "visual"
Requires-Dist: pillow>=11.3.0; extra == "visual"
Requires-Dist: pypng>=0.20220715.0; extra == "visual"
Requires-Dist: shapely>=2.1.2; extra == "visual"
Provides-Extra: doomgen
Requires-Dist: doomgen; extra == "doomgen"
Dynamic: license-file

# COMRAD

COMRAD is a cooperative multi-agent reinforcement learning benchmark built on ViZDoom for first-person visual coordination. This repository contains the benchmark runtime assets, COMRAD-specific environment and model code, a modified Sample Factory training stack, paper artifacts, and the scripts used to produce figures, tables, and diagnostic media.

<p align="center">
  <img src="results/demo_all_checkpoints/comrad_vid_gif_540.gif" alt="COMRAD demo" style="width: 100%; max-width: 960px;">
</p>

## Repository map

| Path | Purpose |
| --- | --- |
| `comrad/` | Benchmark package: training entrypoints, environment registration, wrappers, models, shipped scenario assets, launchers, tests. |
| `sample_factory/` | Modified training engine used by COMRAD. |
| `scripts/` | Figure-generation, result aggregation, WAD rendering, and experiment helper scripts. |
| `results/` | Committed figures, tables, result summaries, and selected media artifacts. |
| `DoomGen/` | Procedural generator used to build and vary scenario WADs. |

Directory-level guides are provided in:

- [`comrad/README.md`](comrad/README.md)
- [`sample_factory/README.md`](sample_factory/README.md)
- [`scripts/README.md`](scripts/README.md)
- [`results/README.md`](results/README.md)

## Setup

COMRAD is packaged through `pyproject.toml` and is intended to be run with `uv`.

```bash
uv sync --all-extras
uv run pre-commit install
```

The project requires Python 3.10 or newer. The package metadata and CI workflow are the authoritative sources for dependency resolution.

## Single-run training

The main training entrypoint is `comrad.train`.

```bash
uv run python -m comrad.train \
  --env=stag_hunt_arena \
  --algo=IPPO \
  --train_for_env_steps=10000 \
  --num_workers=4 \
  --num_envs_per_worker=4 \
  --device=cpu
```

Training outputs are written under `train_dir/<experiment>/`. The main runtime log is:

```bash
tail -n 120 train_dir/<experiment>/sf_log.txt
```

For launcher-owned benchmark sweeps, use `comrad/train_all.py` instead of manually reproducing those long commands.

## Recording and evaluation

High-resolution checkpoint recording:

```bash
uv run python -m comrad.record_video \
  --env=stag_hunt_arena \
  --train_dir=train_dir \
  --experiment=<experiment_name> \
  --load_checkpoint_kind=best \
  --resolution=1280x720
```

Automap/top-down heatmap rendering:

```bash
uv run python -m comrad.record_topdown_heatmap \
  --env=ammo_carrier \
  --train_dir=<train_root> \
  --experiment=<experiment_name> \
  --output_dir=results/videos/ammo_carrier_navigation_heatmap \
  --device=cpu \
  --overwrite
```

Both commands expect a full experiment directory with at least `config.json` and the corresponding checkpoint directory.

## Benchmark launchers

COMRAD ships several paper-facing launcher profiles through `comrad/train_all.py`.

- `benchmark`: full benchmark suite
- `agent_scaling`: Armory Siege agent-count sweep
- `platform_chain_curriculum`: curriculum comparison on Platform Chain
- `qmix_lr_sensitivity_pilot`: Short sensitivity study
- `qmix_lr_sensitivity_full`: Full sensitivity study

Run a launcher profile locally with:

```bash
COMRAD_TRAIN_ALL_PROFILE=benchmark \
uv run python -m sample_factory.launcher.run \
  --run=comrad.train_all \
  --backend=processes \
  --max_parallel=1 \
  --pause_between=1
```

The exact benchmark definitions, algorithms, root directory names, and profile constants live at the top of [`comrad/train_all.py`](comrad/train_all.py).

## Scenario assets and DoomGen

Runtime scenario assets are stored in [`comrad/scenarios/`](comrad/scenarios/README.md) as `.cfg` and `.wad` pairs, plus pre-generated batch directories such as `batch_platform_chain_curriculum/`.

Generation workflows are in `DoomGen/`. The usual split is:

- `DoomGen/examples/benchmark`: procedural generators and geometry logic
- `comrad/scenarios/`: runtime WADs and configs consumed by training and evaluation

If a scenario generator changes, rebuild the corresponding WAD in `DoomGen/` and copy the regenerated artifact back into `comrad/scenarios/` before training or evaluation.

### Using DoomGen as a Python library

DoomGen is declared as an optional dependency in the uv workspace. It is **not** installed by a plain `uv sync`:

```bash
uv sync --extra doomgen      # install doomgen + its deps
# or
uv run --extra doomgen python -m comrad.train ...   # one-off with doomgen available
# or
uv pip install -e DoomGen/   # editable install directly from local checkout
```

After any of these, `import doomgen` works from the root Python environment. Without `--extra doomgen`, the core COMRAD package remains self-contained and DoomGen is not pulled in.

## Tests

Run COMRAD and Sample Factory tests with `uv`:

```bash
uv run pytest comrad/tests -v
uv run pytest sample_factory/tests -v
```

Tests in `comrad/tests/` focus on benchmark logic, launchers, and recorder behavior. Tests in `sample_factory/tests/` cover the modified training stack.

## Outputs and artifacts

- `train_dir/`: experiment directories, configs, checkpoints, logs
- `results/`: committed figures, tables, result summaries, selected media
- `wandb/`: local W&B run state and uploads

See [`results/README.md`](results/README.md) for the structure of the committed artifacts.

## License and upstream assets

The repository is released under the MIT license in [`LICENSE`](LICENSE). Upstream and third-party assets are documented in the repository itself, including the bundled `scripts/wad2image/` tool and the paper's citations to ViZDoom, Sample Factory, HARL, PyMARL, and QPLEX.

## Acknowledgements

We would like to thank all researchers, engineers, advisors involved in this work; and the TU/e Supercomputing Center for providing the infrastructure to run the experiments. The environments are built on top of [ViZDoom](https://github.com/Farama-Foundation/ViZDoom). Our infrastructure is built on top of [Sample Factory](https://github.com/alex-petrenko/sample-factory). Our experiments were managed using [Weights & Biases](https://wandb.ai/). For on-policy actor-critic methods, we implement IPPO and MAPPO from the original implementation in [Sample Factory](https://github.com/alex-petrenko/sample-factory), and HAPPO following the [HARL codebase](https://github.com/PKU-MARL/HARL). For off-policy value decomposition methods, we implement IDQN, VDN, QMIX following the [PyMARL framework](https://github.com/oxwhirl/pymarl), and QPLEX following [official QPLEX reference implementation](https://github.com/wjh720/QPLEX).

<!-- ## Citation -->
