Metadata-Version: 2.4
Name: datenwissenschaften
Version: 2.4.0
Summary: Core library for Retro Speedlab: A high-performance RL toolkit for classic games.
License-Expression: GPL-3.0-only
License-File: LICENSE
Keywords: reinforcement-learning,retro-gaming,stable-baselines3,stable-retro,ai,gymnasium
Author: datenwissenschaften
Author-email: martin.franke@datenwissenschaften.com
Requires-Python: >=3.12,<3.13
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: gymnasium (==1.3.0)
Requires-Dist: itsdangerous (==2.2.0)
Requires-Dist: loguru (==0.7.3)
Requires-Dist: numpy (==2.5.1)
Requires-Dist: opencv-python (==4.13.0.92)
Requires-Dist: pyyaml (==6.0.3)
Requires-Dist: redis (==6.4.0)
Requires-Dist: requests (==2.34.2)
Requires-Dist: sb3-contrib (==2.9.0)
Requires-Dist: stable-baselines3 (==2.9.0)
Requires-Dist: stable-retro (==1.0.1)
Requires-Dist: torch (==2.12.1)
Project-URL: Documentation, https://speedlab.datenwissenschaften.com/getting-started
Project-URL: Homepage, https://speedlab.datenwissenschaften.com
Project-URL: Repository, https://github.com/datenwissenschaften/datenwissenschaften
Description-Content-Type: text/markdown

# datenwissenschaften

[![Python 3.12](https://img.shields.io/badge/Python-3.12-3776AB?logo=python&logoColor=white)](https://www.python.org/)
[![License: GPL-3.0](https://img.shields.io/badge/License-GPL--3.0-blue.svg)](LICENSE)

The reinforcement-learning engine behind [Retro Speedlab](https://github.com/datenwissenschaften/retro-speedlab).

`datenwissenschaften` turns classic-game emulators into reproducible training systems. It provides visual and
state-aware environments, recurrent PPO agents, parallel execution, durable checkpoints, episode
recording, and a live browser dashboard in one focused Python package.

> This repository contains the reusable engine. For game runners, end-to-end examples, and user-facing
> documentation, start with [Retro Speedlab](https://github.com/datenwissenschaften/retro-speedlab).

## Why this engine

- **Exploration for sparse rewards** — multi-input CNN-LSTM PPO combines visual frames, normalized RAM, temporal
  memory, and normalized, clipped, annealed Random Network Distillation (RND).
- **Efficient execution** — vectorized environments, automatic worker selection, CUDA tuning, and CPU fallback.
- **Reliable training runs** — atomic checkpoints, resumable model state, automatic savestates, and `.bk2` replay
  capture.
- **Operational visibility** — a local Vue dashboard reports episode outcomes, reward distributions, environment
  details, PPO parameters, RND progress, and automatic-savestate progress.
- **Game-oriented infrastructure** — ROM discovery, RAM models, state machines, visual encoders, and configurable
  action translation.

## Model choices

| Model | Best suited to | Characteristics |
| --- | --- | --- |
| `RecurrentRNDModel` | Sparse-reward NES games and partially observable state | Automatic visual + RAM inputs, NES-tuned recurrent PPO, LSTM memory, intrinsic RND exploration |
| Custom SB3 model | Experiments that need a standard Stable-Baselines3 algorithm | Integrates through the same builder, trainer, callbacks, and dashboard |

`RecurrentRNDModel` is the recommended starting point for visual agents. RND encourages the policy to visit novel
observations, while its influence decays during training so learned external rewards increasingly drive behavior.
The predictor, fixed target, optimizer, reward statistics, and annealing progress are all preserved in checkpoints.
Environment wrappers always use RGB observations and one emulator step per selected action; these are fixed engine
defaults rather than game-level options.
The default profile uses longer 512-step rollouts, a 512-unit LSTM, `gamma=0.999`, `gae_lambda=0.98`, and a slower
10-million-step RND decay. These settings preserve more temporal context and delayed reward information than the
shorter arcade baseline while retaining conservative PPO updates.

Set `training.savestate_beaten_threshold` to the number of victories required before a training state is marked as
beaten and the next automatic savestate is promoted. Each `<State>.beaten` file stores the current victory count;
the default threshold is `1`. PPO telemetry and automatic-savestate safeguards use extrinsic environment reward only;
RND curiosity remains an exploration bonus for policy updates and does not count as savestate progress. Episodes
restored from an automatic savestate receive that state's `progress` value as their initial reward baseline, then
state rewards and penalties add to or subtract from that baseline.

## Installation

The package requires Python 3.12.

```bash
pip install datenwissenschaften
```

For local development:

```bash
git clone https://github.com/datenwissenschaften/datenwissenschaften.git
cd datenwissenschaften
poetry install
cp config.example.yaml config.yaml
```

## Training dashboard

Enable the dashboard with `ui.enable: true`, then open [http://127.0.0.1:18080](http://127.0.0.1:18080). It refreshes
live training telemetry without interrupting the learner.

Dashboard history is restored from and persisted to Redis. The default Redis URL is
`redis://127.0.0.1:6379/0`, and history keys use the `datenwissenschaften:history` prefix. The `ui` mapping accepts
`enable`, `host`, `port`, `max_episodes`, `redis_url`, and `history_key_prefix`. Snapshots retain the latest
1,000 episodes by default and include summarized totals for discarded episodes; set `max_episodes` to another positive
integer or `null` for unlimited retained rows.
Binding to `0.0.0.0` makes the dashboard reachable on the local network; use that only on a trusted network and open
it through the machine's actual IP address.

## How a run fits together

1. A game package defines RAM structures, training states, rewards, and action translation.
2. The environment factory creates vectorized emulator workers and processed visual observations.
3. A model builder creates or restores the selected policy.
4. The trainer coordinates learning, checkpoints, replay capture, telemetry, and optional uploads.
5. The dashboard exposes the active run without coupling the learner to a separate monitoring service.

## Development

Run Python quality checks:

```bash
ruff check src
black --check src
python -m compileall -q src
```

Build the dashboard assets after changing the Vue frontend:

```bash
cd src/datenwissenschaften/ui/frontend
npm ci
npm run build
```

## License

Copyright © datenwissenschaften contributors. Distributed under the [GNU General Public License v3.0](LICENSE).

