Metadata-Version: 2.4
Name: gymnasium_mars_lander
Version: 1.0.0
Summary: A reinforcement learning environment for the Mars Lander CG puzzle
Author-email: Quentin Deschamps <quentindeschamps18@gmail.com>
License: MIT
Project-URL: Repository, https://github.com/Quentin18/gymnasium-mars-lander
Keywords: Reinforcement Learning,game,RL,AI,gymnasium,pygame
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Games/Entertainment
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: gymnasium==1.2.3
Requires-Dist: pygame==2.6.1
Provides-Extra: training
Requires-Dist: matplotlib==3.10.9; extra == "training"
Requires-Dist: moviepy==2.2.1; extra == "training"
Requires-Dist: rl-zoo3==2.8.0; extra == "training"
Requires-Dist: tensorboard==2.20.0; extra == "training"
Provides-Extra: testing
Requires-Dist: pytest; extra == "testing"
Provides-Extra: quality
Requires-Dist: ruff; extra == "quality"
Requires-Dist: ty; extra == "quality"
Dynamic: license-file

# Gymnasium Mars Lander

[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://pre-commit.com/)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![ty](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ty/main/assets/badge/v0.json)](https://github.com/astral-sh/ty)

Gymnasium environment for the Mars Lander CodinGame puzzles:

- [Mars Lander - Episode 1](https://www.codingame.com/training/easy/mars-lander-episode-1)
- [Mars Lander - Episode 2](https://www.codingame.com/training/easy/mars-lander-episode-2)
- [Mars Lander - Episode 3](https://www.codingame.com/training/easy/mars-lander-episode-3)
- [Mars Lander - Optimization](https://www.codingame.com/multiplayer/optimization/mars-lander)

![Demo](images/demo.gif)

<table>
    <tbody>
        <tr>
            <td>Action Space</td>
            <td><code>Box(-1, 1, (2,), float32)</code></td>
        </tr>
        <tr>
            <td>Observation Space</td>
            <td><code>Box(-1, 1, (12,), float32)</code></td>
        </tr>
        <tr>
            <td>Import</td>
            <td><code>gymnasium.make("gymnasium_mars_lander:gymnasium_mars_lander/MarsLander-v0")</code></td>
        </tr>
    </tbody>
</table>

This package is inspired by the article of Antoine
Broyelle: [Learning to Land on Mars with Reinforcement Learning](https://antoinebrl.github.io/blog/rl-mars-lander/).

## Installation

To install `gymnasium-mars-lander` with pip, execute:

```bash
pip install gymnasium_mars_lander
```

From source:

```bash
git clone https://github.com/Quentin18/gymnasium-mars-lander
cd gymnasium-mars-lander/
pip install -e .
```

For running on CPU with extras:

```bash
pip install -e .[training,testing,quality] --extra-index-url https://download.pytorch.org/whl/cpu
```

## Environment

### Action Space

The action is a `ndarray` with two continuous variables:

- The rotation change between -15 and 15 degrees.
- The thrust change between -1 and 1.

The values are normalized between -1 and 1.

### Observation Space

The observation is a `ndarray` of 12 continuous variables:

- The distances in six directions from the current position.
- The rower horizontal and vertical speed, angle and thrust.
- The horizontal and vertical distances to the middle of the landing area.

The values are normalized between -1 and 1.

The following figure shows the sensors used:

![Sensors](images/sensors.png)

### Rewards

The rewards are described by the following table:

| Condition                                                            | Reward                                 |
|----------------------------------------------------------------------|----------------------------------------|
| The rover leaves the frame                                           | `-150`                                 |
| The rover runs out of fuel                                           | `-150`                                 |
| The rover crashes outside flat ground with incorrect angle and speed | `-100`                                 |
| The rover crashes with correct angle and speed                       | `-75`                                  |
| The rover crashes on flat ground                                     | `-50`                                  |
| The rover approaches the landing area                                | `0.01`                                 |
| The rover lands successfully                                         | `200` + Amount of remaining propellant |

### Starting State

The starting state is generated by choosing a random CodinGame test case.
When the `eval_env` argument is `False`, some random augmentations are applied to the test case.
For each test case, there are five starting positions in increasing order of difficulty.
The starting position can be set with the `start` argument.

The following figure shows the starting positions:

![Starts](images/starts.png)

### Episode End

The episode ends if either of the following happens:

1. Termination: The rower lands on the landing area or runs out of fuel or crashes.
2. Truncation: Episode length is greater than 2000.

### Arguments

- `episode`: episode number between 1 and 3. The default value is `2`.
- `start`: starting position between -1 and 4. The default value is `-1`.
- `eval_env`: if `True`, the random augmentations are disabled. The default value is `False`.
- `sequential_maps`: if `True`, the maps are generated sequentially. The default value is `False`.

```python
import gymnasium as gym

gym.make(
    "gymnasium_mars_lander:gymnasium_mars_lander/MarsLander-v0",
    episode=2,
    start=-1,
    eval_env=False,
    sequential_maps=False,
)
```

## Trained agents

There is one trained agent for each episode:

| Path                                                                         | Episode |
|------------------------------------------------------------------------------|---------|
| `rl-trained-agents/ppo/gymnasium_mars_lander-MarsLander-v0_1/best_model.zip` | 1       |
| `rl-trained-agents/ppo/gymnasium_mars_lander-MarsLander-v0_2/best_model.zip` | 2       |

## Usage

You can use [RL Baselines3 Zoo](https://github.com/DLR-RM/rl-baselines3-zoo) to train and evaluate agents:

```bash
pip install rl_zoo3
```

### Train an Agent

The hyperparameters are defined in `hyperparams/ppo.yml`.

To train a PPO agent for the Mars Lander game, execute:

```bash
python -m rl_zoo3.train \
  --algo ppo \
  --env gymnasium_mars_lander/MarsLander-v0 \
  --tensorboard-log logs \
  --trained-agent rl-trained-agents/ppo/gymnasium_mars_lander-MarsLander-v0_2/best_model.zip \
  --n-timesteps 10000000 \
  --log-interval 100 \
  --eval-freq 10000 \
  --eval-episodes 100 \
  --seed 42 \
  --gym-packages gymnasium_mars_lander \
  --conf-file hyperparams/ppo.yml \
  --progress \
  --env-kwargs "episode:int(2)" "start:int(-1)" "sequential_maps:True"
```

### Enjoy a Trained Agent

To see a trained agent in action on random test cases, execute:

```bash
python -m rl_zoo3.enjoy \
  --algo ppo \
  --env gymnasium_mars_lander/MarsLander-v0 \
  --n-timesteps 1000 \
  --deterministic \
  --seed 42 \
  --gym-packages gymnasium_mars_lander \
  --load-best \
  --progress \
  --env-kwargs "episode:int(2)" "start:int(-1)" "sequential_maps:True"
```

To see a trained agent in action on CodinGame test cases, execute:

```bash
python -m scripts.enjoy --path rl-trained-agents/ppo/gymnasium_mars_lander-MarsLander-v0_2/best_model.zip
```

To record videos of a trained agent in action on CodinGame test cases, execute:

```bash
python -m scripts.enjoy \
  --path rl-trained-agents/ppo/gymnasium_mars_lander-MarsLander-v0_2/best_model.zip \
  --record-video
```

## Tests

To run tests, execute:

```bash
pytest
```

## Citing

To cite the repository in publications:

```bibtex
@misc{gymnasium-mars-lander,
  author = {Quentin Deschamps},
  title = {Gymnasium Mars Lander},
  year = {2026},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Quentin18/gymnasium-mars-lander}},
}
```

## References

- [Gymnasium](https://github.com/Farama-Foundation/Gymnasium)
- [RL Baselines3 Zoo](https://github.com/DLR-RM/rl-baselines3-zoo)
- [Stable Baselines3](https://github.com/DLR-RM/stable-baselines3)
- [Mars Lander with Reinforcement Learning](https://github.com/antoinebrl/rl-mars-lander)
- [CodinGame Forum - Mars Lander Puzzle discussion](https://www.codingame.com/forum/t/mars-lander-puzzle-discussion/32/338)

## Author

[Quentin Deschamps](mailto:quentindeschamps18@gmail.com)
