Metadata-Version: 2.1
Name: dm-meltingpot
Version: 2.4.1
Summary: A suite of test scenarios for multi-agent reinforcement learning.
Home-page: https://github.com/google-deepmind/meltingpot
Download-URL: https://github.com/google-deepmind/meltingpot/releases
Author: DeepMind
Author-email: noreply@google.com
License: Apache 2.0
Keywords: multi-agent reinforcement-learning python machine-learning
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Provides-Extra: dev
License-File: LICENSE

# Melting Pot

*A suite of test scenarios for multi-agent reinforcement learning.*

<!-- linter off -->

<!-- linter on -->

<!-- disableFinding(SNIPPET_INVALID_LANGUAGE) -->
<!-- disableFinding(IMAGE_ALT_TEXT_INACCESSIBLE) -->

<div align="center">
  <img src="https://github.com/google-deepmind/meltingpot/blob/main/docs/images/meltingpot_montage.gif?raw=true"
       alt="Melting Pot substrates"
       height="250" width="250" />
</div>

[Melting Pot 2.0 Tech Report](https://arxiv.org/abs/2211.13746)
[Melting Pot Contest at NeurIPS 2023](https://www.aicrowd.com/challenges/meltingpot-challenge-2023)

## About

Melting Pot assesses generalization to novel social situations involving both
familiar and unfamiliar individuals, and has been designed to test a broad range
of social interactions such as: cooperation, competition, deception,
reciprocation, trust, stubbornness and so on. Melting Pot offers researchers a
set of over 50 multi-agent reinforcement learning _substrates_ (multi-agent
games) on which to train agents, and over 256 unique test _scenarios_ on which
to evaluate these trained agents. The performance of agents on these held-out
test scenarios quantifies whether agents:

*   perform well across a range of social situations where individuals are
    interdependent,
*   interact effectively with unfamiliar individuals not seen during training

The resulting score can then be used to rank different multi-agent RL algorithms
by their ability to generalize to novel social situations.

We hope Melting Pot will become a standard benchmark for multi-agent
reinforcement learning. We plan to maintain it, and will be extending it in the
coming years to cover more social interactions and generalization scenarios.

If you are interested in extending Melting Pot, please refer to the
[Extending Melting Pot](https://github.com/google-deepmind/meltingpot/blob/main/docs/extending.md) documentation.

## Installation

Melting Pot is available on PyPI](https://pypi.python.org/pypi/dm-meltingpot)
and can be installed using:

```shell
pip install dm-meltingpot
```

After doing this you can then `import meltingpot` in your own code.

NOTE: Melting Pot is built on top of [DeepMind Lab2D](https://github.com/google-deepmind/lab2d)
which is distributed as pre-built wheels. If there is no appropriate wheel for
`dmlab2d`, you will need to build it from source (see
[the `dmlab2d` `README.md`](https://github.com/google-deepmind/lab2d/blob/main/README.md)
for details).

## Development

### Codespace

The easiest way to work on the Melting Pot source code, is to use our
pre-configured development environment via a
[Github CodeSpace](https://github.com/features/codespaces).

This provides a tested development workflow that allows for reproducible builds,
and minimizes dependency management. We strongly advise preparing all Pull
Requests for Melting Pot via this workflow.

### Manual setup

If you want to work on the Melting Pot source code within your own development
environment you will have to handle installation and dependency management
yourself.

For example, you can perform an editable installation as follows:

1.  Clone Melting Pot:

    ```shell
    git clone -b main https://github.com/google-deepmind/meltingpot
    cd meltingpot
    ```

2.  Create and activate a virtual environment:

    ```shell
    python -m venv venv
    source venv/bin/activate
    ```

3.  Install Melting Pot:

    ```shell
    pip install --editable .[dev]
    ```

4.  Test the installation:

    ```shell
    pytest --pyargs meltingpot
    ```

## Example usage

### Evaluation
The [evaluation](https://github.com/google-deepmind/meltingpot/blob/main/meltingpot/utils/evaluation/evaluation.py) library can be used
to evaluate [SavedModel](https://www.tensorflow.org/guide/saved_model)s
trained on Melting Pot substrates.

Evaluation results from the [Melting Pot 2.0 Tech Report](https://arxiv.org/abs/2211.13746)
can be viewed in the [Evaluation Notebook](https://github.com/google-deepmind/meltingpot/blob/main/notebooks/evaluation_results.ipynb).

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/deepmind/meltingpot/blob/main/notebooks/evaluation_results.ipynb)

### Interacting with the substrates

You can try out the substrates interactively with the
[human_players](https://github.com/google-deepmind/meltingpot/blob/main/meltingpot/human_players) scripts. For example, to play
the `clean_up` substrate, you can run:

```shell
python meltingpot/human_players/play_clean_up.py
```

You can move around with the `W`, `A`, `S`, `D` keys, Turn with `Q`, and `E`,
fire the zapper with `1`, and fire the cleaning beam with `2`. You can switch
between players with `TAB`. There are other substrates available in the
[human_players](https://github.com/google-deepmind/meltingpot/blob/main/meltingpot/human_players) directory. Some have multiple
variants, which you select with the `--level_name` flag.

## Documentation

Full documentation is available [here](https://github.com/google-deepmind/meltingpot/blob/main/docs/index.md).

## Citing Melting Pot

If you use Melting Pot in your work, please cite the accompanying articles:

```bibtex
@inproceedings{leibo2021meltingpot,
    title={Scalable Evaluation of Multi-Agent Reinforcement Learning with
           Melting Pot},
    author={Joel Z. Leibo AND Edgar Du\'e\~nez-Guzm\'an AND Alexander Sasha
            Vezhnevets AND John P. Agapiou AND Peter Sunehag AND Raphael Koster
            AND Jayd Matyas AND Charles Beattie AND Igor Mordatch AND Thore
            Graepel},
    year={2021},
    journal={International conference on machine learning},
    organization={PMLR},
    url={https://doi.org/10.48550/arXiv.2107.06857},
    doi={10.48550/arXiv.2107.06857}
}
```

```bibtex
@article{agapiou2022melting,
  title={Melting Pot 2.0},
  author={Agapiou, John P and Vezhnevets, Alexander Sasha and Du{\'e}{\~n}ez-Guzm{\'a}n, Edgar A and Matyas, Jayd and Mao, Yiran and Sunehag, Peter and K{\"o}ster, Raphael and Madhushani, Udari and Kopparapu, Kavya and Comanescu, Ramona and Strouse, {DJ} and Johanson, Michael B and Singh, Sukhdeep and Haas, Julia and Mordatch, Igor and Mobbs, Dean and Leibo, Joel Z},
  journal={arXiv preprint arXiv:2211.13746},
  year={2022}
}
```

## Disclaimer

This is not an officially supported Google product.
