Metadata-Version: 2.4
Name: TrackmaniaRL
Version: 1.0.2
Summary: Reinforcement-learning library for training agents in Trackmania 2020
Author: Jakub Szulc
License-Expression: MIT
Project-URL: Homepage, https://github.com/Palamabron/AITrackmania
Project-URL: Repository, https://github.com/Palamabron/AITrackmania
Project-URL: Documentation, https://github.com/Palamabron/AITrackmania/tree/main/readme
Project-URL: Issues, https://github.com/Palamabron/AITrackmania/issues
Project-URL: Changelog, https://github.com/Palamabron/AITrackmania/blob/main/CHANGELOG.md
Project-URL: Security, https://github.com/Palamabron/AITrackmania/blob/main/SECURITY.md
Keywords: reinforcement learning,robot learning,trackmania,self driving,roborace
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Games/Entertainment
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: pydantic>=2.0
Requires-Dist: numpy>=1.24
Requires-Dist: torch>=2.4
Requires-Dist: tensordict>=0.6
Requires-Dist: gymnasium>=0.29
Requires-Dist: PyYAML>=6.0
Provides-Extra: orchestrator
Requires-Dist: google-genai>=2.2.0; extra == "orchestrator"
Requires-Dist: optuna>=3.0; extra == "orchestrator"
Provides-Extra: algorithms
Requires-Dist: einops>=0.7; extra == "algorithms"
Requires-Dist: gymnasium>=0.29; extra == "algorithms"
Requires-Dist: loguru>=0.7; extra == "algorithms"
Requires-Dist: pandas>=2.0; extra == "algorithms"
Requires-Dist: torchrl>=0.11; extra == "algorithms"
Provides-Extra: trackmania
Requires-Dist: gymnasium>=0.29; extra == "trackmania"
Requires-Dist: vgamepad>=0.1.0; sys_platform == "win32" and extra == "trackmania"
Provides-Extra: wandb
Requires-Dist: wandb>=0.15.8; extra == "wandb"
Provides-Extra: distributed
Requires-Dist: grpcio>=1.66; extra == "distributed"
Requires-Dist: protobuf>=5.27; extra == "distributed"
Requires-Dist: safetensors>=0.4.5; extra == "distributed"
Requires-Dist: zstandard>=0.23; extra == "distributed"
Requires-Dist: vgamepad>=0.1.0; sys_platform == "win32" and extra == "distributed"
Provides-Extra: explain
Requires-Dist: captum>=0.7; extra == "explain"
Provides-Extra: vision
Requires-Dist: torchvision>=0.15; extra == "vision"
Dynamic: license-file

# TrackmaniaRL

[![PyPI](https://img.shields.io/pypi/v/TrackmaniaRL?color=blue)](https://pypi.org/project/TrackmaniaRL/)
[![Python](https://img.shields.io/pypi/pyversions/TrackmaniaRL?color=3776AB)](https://pypi.org/project/TrackmaniaRL/)
[![CI](https://github.com/Palamabron/AITrackmania/actions/workflows/ci.yml/badge.svg)](https://github.com/Palamabron/AITrackmania/actions/workflows/ci.yml)
[![License](https://img.shields.io/pypi/l/TrackmaniaRL?color=green)](https://github.com/Palamabron/AITrackmania/blob/main/LICENSE)
[![Status](https://img.shields.io/pypi/status/TrackmaniaRL?color=orange)](https://pypi.org/project/TrackmaniaRL/)

TrackmaniaRL is a reinforcement-learning library for training agents in
Trackmania 2020. It combines ready-to-use algorithms, replay buffers, model
families and Trackmania telemetry with explicit interfaces for replacing any
component in an experiment.

The current release is available on
[PyPI](https://pypi.org/project/TrackmaniaRL/). TrackmaniaRL requires Python
3.12 or newer.

## What you get

- asynchronous local or distributed actor/learner training;
- SAC, REDQ-SAC, TQC, IQN and stable discrete SAC learners;
- uniform, prioritized, sequence and demonstration-mixing replay;
- typed configuration, transitions and training batches;
- Trackmania telemetry, lidar and track-geometry feature pipelines;
- durable rollout journals, safe policy transfer and resumable checkpoints;
- local JSONL observability with optional W&B, Captum, Gemini and Optuna
  integrations;
- an installable extension project generated by `trackmaniarl init`.

TrackmaniaRL has no global runtime configuration and no mandatory external
tracker. A run is described by `run.yaml` and explicit `module:attribute`
component paths.

## Install and create an agent

Install the published CLI with [uv](https://docs.astral.sh/uv/):

```bash
uv tool install trackmaniarl
trackmaniarl init my-trackmania-agent --template trackmania
cd my-trackmania-agent
uv sync
uv run trackmaniarl validate run.yaml
```

The `trackmania` template creates a commented, installable agent project with
the Trackmania, algorithm, distributed and W&B extras declared for you. Omit
`--template trackmania` to generate the smaller, game-free starter project.
`trackmaniarl validate` checks imports, contracts and a synthetic learner update
without starting the game or contacting an external tracker.

To add the SDK to an existing Python project instead, choose only the extras you
need:

```bash
uv add trackmaniarl
uv add "trackmaniarl[algorithms,distributed]"
```

| Extra | Adds |
| --- | --- |
| `algorithms` | TorchRL-based algorithm dependencies |
| `trackmania` | Trackmania environment and Windows virtual-gamepad support |
| `distributed` | authenticated gRPC rollouts, safetensors and compression |
| `wandb` | Weights & Biases logging |
| `explain` | Captum attribution helpers |
| `orchestrator` | Gemini and Optuna experiment strategies |
| `vision` | torchvision support |

## Run Trackmania

Live collection requires Trackmania 2020 on Windows, the bundled OpenPlanet
plugin and a prepared map/geometry asset. Follow the
[Trackmania workflow](https://github.com/Palamabron/AITrackmania/blob/main/readme/trackmania.md)
or the concrete
[agent OpenPlanet guide](https://github.com/Palamabron/AITrackmania/blob/main/my-trackmania-agent/openplanet/README.md)
before starting the game integration.

The generated Trackmania project pins the patched
[Palamabron/vgamepad](https://github.com/Palamabron/vgamepad/tree/5f3435df3f8a0e658feb58b207d9137cdb5183cd)
revision containing the unreleased Windows installation fix from
[vgamepad PR #47](https://github.com/yannbouteiller/vgamepad/pull/47). Keep
that source pin until the fix is included in an upstream vgamepad release.

With Trackmania and the OpenPlanet plugin running:

```bash
uv run trackmaniarl track check
uv run trackmaniarl smoke run.yaml --transitions 100
uv run trackmaniarl train run.yaml
```

The bounded smoke test uses the same asynchronous learner/actor path as
training, verifies a live policy refresh and writes a checkpoint. Start a fresh
run directory when the run API or immutable configuration changes; the current
schema is RunSpec `1.2`.

On Windows, a generated project selects the tested CUDA PyTorch wheels. Linux
uses CPU wheels by default and can host an offline or remote learner. ROCm users
must select the matching AMD Torch index; macOS uses the normal PyPI wheel and
can use MPS. `device: auto` resolves CUDA, ROCm, MPS or CPU from the installed
Torch build.

## Runtime model

```text
run.yaml -> coordinator/learner -> SQLite WAL -> replay -> update -> checkpoint
              ^       |
              |       +---- safetensors policy snapshot
              |
              +---- local or remote actors -> durable rollout spool
```

`trackmaniarl train` starts a coordinator/learner and one local actor as
independent, Windows-safe `spawn` processes. Collection continues while the
learner updates replay and periodically publishes policy snapshots.

For multiple machines, set the same `TRACKMANIARL_DISTRIBUTED_TOKEN` on every
participant and expose the learner through an encrypted tunnel. The learner
binds to loopback so its bearer token and rollout data are not sent over the
network in clear text:

```bash
# training machine
uv run trackmaniarl learner run.yaml --bind 127.0.0.1:8787

# Trackmania machine: create the tunnel first
ssh -N -L 8787:127.0.0.1:8787 TRAINING_MACHINE
uv run trackmaniarl actor run.yaml --connect 127.0.0.1:8787 --actor-id PC-1
```

The handshake rejects mismatched run fingerprints, map UIDs, geometry hashes
and feature/action contracts. Rollouts use Protobuf/gRPC with Zstandard
compression, and policy state is transferred with safetensors rather than
pickle.

## Components and extension API

`trackmaniarl.builtins` is the supported catalogue of bundled algorithms,
models, feature pipelines and replay strategies. A component can also be
referenced directly, for example:

```yaml
components:
  learner:
    class_path: trackmaniarl.algorithms.implicit_quantile_q_learning:ImplicitQuantileQLearning
```

The stable contracts in `trackmaniarl.core` include `Learner`, `Policy`,
`ModelFactory`, `ReplayStore`, `Sampler`, `FeaturePipeline`, `Evaluator`,
`RunLogger` and `CheckpointCodec`. Game-specific implementations belong in the
generated extension project, so offline validation does not require Trackmania
or optional game dependencies.

Every run writes a redacted immutable manifest, local JSONL events, checkpoints
and bounded compressed episode artifacts. Only the learner needs W&B
credentials; `WANDB_API_KEY` can be supplied through the environment or project
`.env`.

See the [SDK guide](https://github.com/Palamabron/AITrackmania/blob/main/readme/sdk.md)
for the full component schema and a built-in run example. Release history is in
the [changelog](https://github.com/Palamabron/AITrackmania/blob/main/CHANGELOG.md).

## Development

Clone the repository and install the development group:

```bash
git clone https://github.com/Palamabron/AITrackmania.git
cd AITrackmania
uv sync --group dev
uv run poe fmt
uv run poe types
uv run poe test
```

The commands are intentionally identical on Windows, Linux, WSL and CI. See
[CONTRIBUTING.md](https://github.com/Palamabron/AITrackmania/blob/main/CONTRIBUTING.md)
and [SECURITY.md](https://github.com/Palamabron/AITrackmania/blob/main/SECURITY.md)
before opening a contribution or reporting a vulnerability.

## Project status and attribution

TrackmaniaRL is beta software. The project originated from TMRL and has since
been substantially redesigned. It is not affiliated with or endorsed by
Ubisoft, Nadeo or the TMRL maintainers. Trackmania is a trademark of
Nadeo/Ubisoft. See [NOTICE](https://github.com/Palamabron/AITrackmania/blob/main/NOTICE)
for attribution.
