Retro Speedlab Library

Reinforcement-learning engine for training recurrent agents on classic video games.

The reusable reinforcement-learning engine behind Retro Speedlab.

It combines visual observations, emulator RAM, recurrent PPO, Random Network Distillation (RND), vectorized environments, resumable training, replay capture, and live telemetry in a reproducible Python training system.

Highlights

Recurrent CNN-LSTM PPO for partially observable environments
Multi-input policies combining RGB frames and emulator RAM
Adaptive Random Network Distillation (RND) for sparse-reward exploration
Parallel vectorized emulator environments
Automatic worker selection, CUDA tuning, and CPU fallback
Atomic, resumable checkpoints including exploration state
.bk2 episode recording
Live Vue-based training telemetry
Reusable abstractions for RAM, game states, rewards, and actions

Looking for game runners, end-to-end examples, and user-facing documentation? Start with Retro Speedlab.

Architecture

A training run combines game-specific definitions with reusable environment, model, training, persistence, and monitoring components.

                         Retro Speedlab
                              │
                    ┌─────────▼─────────┐
                    │  Game Definition  │
                    │                   │
                    │ RAM · Rewards ·   │
                    │ States · Actions  │
                    └─────────┬─────────┘
                              │
                    ┌─────────▼─────────┐
                    │ Vectorized Retro  │
                    │   Environments    │
                    └─────────┬─────────┘
                              │
                 ┌────────────┴────────────┐
                 │                         │
             RGB Frames                 RAM State
                 │                         │
            CNN Encoder                 Encoder
                 │                         │
                 └────────────┬────────────┘
                              │
                             LSTM
                              │
                    ┌─────────▼─────────┐
                    │  Recurrent PPO    │
                    │      + RND        │
                    └─────────┬─────────┘
                              │
               ┌──────────────┼──────────────┐
               │              │              │
          Checkpoints      Replays       Telemetry
                                             │
                                     ┌───────▼───────┐
                                     │ Live Dashboard │
                                     └───────────────┘

At runtime:

A game package defines RAM structures, training states, rewards, and action translation.
The environment factory creates vectorized emulator workers and processed visual observations.
A model builder creates or restores the selected policy.
The trainer coordinates learning, checkpoints, replay capture, telemetry, and optional uploads.
The dashboard exposes the active run without coupling the learner to a separate monitoring service.

Why this engine

Exploration for sparse rewards

AdaptiveRecurrentRNDModel combines visual frames, normalized RAM, temporal memory, and recurrent PPO with normalized, clipped, and annealed Random Network Distillation.

RND provides an intrinsic reward for novel observations, encouraging exploration when useful external rewards are rare. Its influence decays during training so that learned external rewards increasingly determine policy behavior.

Temporal and state-aware policies

Visual observations are processed alongside structured emulator RAM. An LSTM maintains temporal context for environments where the current observation alone does not fully describe the game state.

Environment wrappers use RGB observations and one emulator step per selected action as fixed engine defaults rather than game-level options.

Efficient execution

Training supports vectorized environments, automatic worker selection, CUDA tuning, and CPU fallback.

The engine is designed to separate game-specific definitions from reusable training infrastructure, allowing the same training pipeline to operate across different environments.

Reliable, resumable training

Training state is persisted through atomic checkpoints.

For the adaptive recurrent RND model, checkpoints preserve:

policy state
RND predictor
fixed RND target
optimizer state
reward statistics
adaptation state
annealing progress

This allows interrupted experiments to resume without silently resetting the exploration process.

Operational visibility

Training telemetry is exposed through a local browser dashboard without requiring a separate monitoring service.

Episode recordings can additionally be persisted as .bk2 replays for later inspection.

Models

`AdaptiveRecurrentRNDModel`

The recommended model for sparse-reward games and partially observable environments.

It combines:

visual CNN encoding
normalized emulator RAM
LSTM temporal memory
recurrent PPO
adaptive intrinsic RND exploration

The model automatically configures parameters including score-staleness windows, missing-win windows, exploration multipliers, entropy, learning rate, clip range, and RND update pressure using characteristics of the action space, rollout size, training horizon, fitness volatility, score staleness, and win staleness.

The default profile uses:

Parameter	Default
Rollout length	`512` steps
LSTM size	`256` units
`gamma`	`0.999`
`gae_lambda`	`0.98`
RND decay horizon	`5,000,000` steps

These defaults preserve more temporal context and delayed reward information than a shorter arcade-oriented baseline while retaining conservative PPO updates.

Custom Stable-Baselines3 model

Experiments requiring a standard Stable-Baselines3 algorithm can integrate through the same model builder, trainer, callbacks, and dashboard infrastructure.

Model	Best suited to
`AdaptiveRecurrentRNDModel`	Sparse-reward games and partially observable state
Custom SB3 model	Experiments using standard Stable-Baselines3 algorithms

Installation

Retro Speedlab Library requires Python 3.12.

Install the published package:

pip install datenwissenschaften

For local development:

git clone https://github.com/datenwissenschaften/retro-speedlab-library.git
cd retro-speedlab-library

poetry install
cp config.example.yaml config.yaml

The Python package is currently published as datenwissenschaften.

Training dashboard

Enable the dashboard in the configuration:

ui:
  enable: true

Then open:

http://127.0.0.1:18080

The dashboard exposes live training telemetry without interrupting the learner, including:

episode outcomes
reward distributions
environment details
PPO parameters
RND progress

Dashboard history and non-file training state are persisted to Redis. This includes best-episode references and metrics, callback state, and target memory.

Best episodes are scoped independently by game identity and game savestate, for example:

level1-1

Model checkpoints and .bk2 episode recordings remain on disk.

The default Redis connection is:

redis://127.0.0.1:6379/0

History keys use:

datenwissenschaften:history

The ui configuration accepts:

ui:
  enable: true
  host: 127.0.0.1
  port: 18080
  max_episodes: 1000
  redis_url: redis://127.0.0.1:6379/0
  history_key_prefix: datenwissenschaften:history

Snapshots retain the latest 1,000 episodes by default and include summarized totals for discarded episodes.

Set max_episodes to another positive integer or null for unlimited retained rows.

Binding the dashboard to 0.0.0.0 makes it reachable from other machines on the local network. Use this only on a trusted network.

Project responsibilities

The library focuses on the reusable reinforcement-learning engine rather than individual game implementations.

Its responsibilities include:

Environment
├── emulator integration
├── vectorized workers
├── visual observations
└── action translation

Game state
├── RAM models
├── state machines
├── rewards
└── training objectives

Learning
├── CNN encoding
├── recurrent PPO
├── LSTM memory
└── RND exploration

Training
├── model construction
├── checkpointing
├── callbacks
├── replay capture
└── telemetry

Monitoring
├── episode history
├── training metrics
├── Redis persistence
└── Vue dashboard

Game runners, end-to-end examples, and higher-level project documentation live in Retro Speedlab.

Development

Install the development environment:

poetry install

Run the Python quality checks:

ruff check src
black --check src
python -m compileall -q src

After modifying the Vue dashboard frontend, rebuild its assets:

cd src/datenwissenschaften/ui/frontend
npm ci
npm run build

License

Distributed under the GNU General Public License v3.0.

Metadata

Release files for datenwissenschaften 2.10.12

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for datenwissenschaften 2.10.12
File	Size	Uploaded
datenwissenschaften-2.10.12.tar.gz	138.5 kB	Sep 20, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for datenwissenschaften 2.10.12
File	Interpreter	ABI	Platform	Reset
datenwissenschaften-2.10.12-py3-none-any.whl 160.8 kB Sep 20, 2026	Python 3	none	any	Details

Total release size: 299.3 kB

Release files / datenwissenschaften-2.10.12.tar.gz

Download URL	datenwissenschaften-2.10.12.tar.gz
Size	138.5 kB
Tags	Source
SHA-256 checksum How to use checksums	`561ec1bec000f148bc3c6e49a7a26dfdca29aa5b18856399007c07c00cdd327c`
BLAKE2b-256 checksum How to use checksums	`dcfbe840310e625c427f527ea76e59cea0699c39a112da237f7f312ac19cb9aa`
Upload date	Sep 20, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/2.5.1 CPython/3.12.14 Linux/6.17.0-1022-azure`

Release files / datenwissenschaften-2.10.12-py3-none-any.whl

Download URL	datenwissenschaften-2.10.12-py3-none-any.whl
Size	160.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`9be9f02520220bc1374d8738e095e2c646833da8e7aa0f24b30f690893ddd1e6`
BLAKE2b-256 checksum How to use checksums	`e05781262b038cf7094e0bd9c4f71f026e3bd8c2684638c37e6b7d2a8ecc4c36`
Upload date	Sep 20, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/2.5.1 CPython/3.12.14 Linux/6.17.0-1022-azure`