Retro Speedlab Core

Tool-assisted speedrun search for classic video games. The engine searches the fastest movie of buttons to press on every frame from power-on, verifies that it replays, and streams the run to Retro Speedlab. Game packages supply the game knowledge (RAM map, states, rewards, actions); this repository turns it into a speedrun.

Highlights

• Power-on start like a real speedrun: every movie boots the game at its title screen; levels are states of one full-game run
• Level by level: the search stays on the earliest unbeaten level until it is beaten, starting where the route of the level before it ends, then moves on
• Three search modes per segment on many emulator processes: Go-Explore walks, a breadth-first frontier search over short input combinations and a Monte Carlo tree search
• Learned guidance: RAM values never seen before, a move table learned from every stored piece, and a PPO policy that trains on the GPU
• Every segment is verified: the whole movie replays from power-on before a new segment counts
• Route memory from the practised levels guides the search towards each level's exit
• .bk2 movie and MP4 video of the verified run
• Live relay of the newest verified segment, the run's time, level splits and the lab's status to the Retro Speedlab backend
• Lab runs: when the search is stuck or stalls, Claude Code improves the game package on its own branch

Architecture

flowchart TD
    Game["Game package<br/>RAM · States · Rewards · Actions"]
    Env["Stable Retro emulator<br/>StateMachineGymWrapper"]
    Search["Segment search<br/>one worker per process"]
    Movie["Verified movie<br/>progress.msgpack · tas.bk2 · screen.mp4"]
    Relay["Live relay<br/>snapshot · whole movie"]
    Lab["Lab run<br/>agent/daily.sh"]

    Game --> Env
    Env --> Search
    Search --> Movie
    Movie --> Relay
    Movie -- "stuck or stalled" --> Lab
    Lab --> Game

Speedrun search

The app's default workflow is speedrun: it writes the fastest movie from power-on for any game package and improves it forever:

python -m datenwissenschaften.speedrun.run search <wrapper> <config> <until-state>
python -m datenwissenschaften.speedrun.run improve <wrapper> <config>
python -m datenwissenschaften.speedrun.run verify <wrapper> <config> <progress>

The run is split at the state machine's transitions. Each segment is searched on speedrun.workers emulator processes over an archive that keeps the fastest savestate per cell (state, 8 px position, height, airborne and RamInfo.dynamics(), the game's coarse view of whatever moves nearby). Three modes share each search by time:

• Go-Explore walks (60 %) branch from rarely visited cells with short input bursts; half the time they start from the tenth of cells that earned the most of the segment's reward.
• Frontier search (20 %) tries every action with holds of 1, 3 and 8 decisions, three deep and breadth first, from a chosen cell: the frame-precise jumps random walks rarely hit.
• Monte Carlo tree search (20 %) expands the moves with the best upper confidence bound and scores short rollouts by reward and new cells.

Cells gain weight for RAM byte values never seen before in the segment and for being close to the exit on the route memory. Walks and rollouts follow a move table learned from every stored piece (cache/tas/prior.msgpack) 30 % of the time and the PPO policy 20 % of the time. A full archive drops its least promising cell. Each search process keeps 1000 cells of the last 12 starts it explored, half with the most reward and half the rarest, and saves them to cache/tas/archives/<start>/<process>.msgpack: a later search from the same start goes on where the last one stopped, and after a restart the processes' files are merged. Before it searches a state, the TAS replays the hints for it from each of its starts in that state: the game package's hints.json (written by the lab, [{"state": ..., "presses": [[frames, [buttons]]]}]) and the shortest way out the PPO policy found (cache/tas/policy_exits.msgpack); a hint that crosses and holds becomes the next piece. A crossing into a later state ends a branch and counts only when the new state holds; deaths and backward transitions drop it. Of the three fastest distinct crossings, the one whose next segment is also fastest is kept, and frames it does not need are shaved off. A segment gets speedrun.segment_seconds per round and twice as long with each retry. Random numbers live in the savestates, so the search plans against the real RNG and can wait frames to change it. A segment without a way out is marked stuck in cache/tas/stuck.json (in practice since the state's first failed search without a find, across the other levels' turns, cache/tas/failures.json) and the way into it is searched again; a search that only ends in a checkpoint counts as one without a way out too. Checkpoints in a state come only within its first turn of searches since a piece last left it (practice: one level's turn, otherwise 3 searches): a chain of checkpoints that has not led out by then is dropped down to the last piece. Going back drops every checkpoint in a row down to the last piece as well, and either way that level gets no new checkpoint in the state it went back from until a piece leaves that state (finding the way back into it is not enough), also after a restart (cache/tas/barred.json). A checkpoint is the reached cell with the fewest frames left on the route map: from a start on the map that cell must be at least 60 frames closer to the exit, from a start off the map any reached map cell will do; otherwise it is the cell with the most reward. A checkpoint on the map always moves closer to the exit, so it is also kept after the first turn; of several workers' checkpoints the one closest to the exit on the map wins, then the most reward.

After every segment the whole movie is replayed from power-on; segments that do not reproduce are searched again. cache/tas/progress.msgpack lets a run resume, cache/tas/tas.bk2 is the verified movie, screen.mp4 its video and search.log the search's log. metadata.speedrun tells the stream the run's time from power-on and from Start, its segments, the level splits (from GameWrapper.levels, with the part before the first level split under the first state's name, opening), each state's goal, the game and its record sources, the engine's events, milestones and time history (journal.json) and the segment being searched. While a segment is searched every worker writes its cells and frames to live/<seed>.msgpack every 2 s and its latest frame at half size to live/<seed>.jpg once a second; the main process folds the reports into search every 5 s and the relay puts a mosaic of the first nine workers' frames (seed order, row by row, seeds in X-Seeds) to /live/attempts.jpg once a second. Whenever the verified movie changes a background thread renders it from power-on to an H.264 MP4 at twice the NES resolution, puts it to /live/movie.mp4 in parts, puts the game memory sampled every 6 frames of the power-on replay to /live/movie-ram.json and publishes speedrun.movie: its version (the first 12 hex digits of its SHA-256), frames, seconds, frame rate and each segment's state, the state it reaches and its start frame, so the stream plays the whole run in sync with its splits. When improve finds a faster segment, the old and the new piece are rendered side by side from the same savestate, put to /live/compare.mp4 and published as speedrun.compare with the state, both frame counts and the time. improve searches every segment again, bounded by its current frames; a faster crossing replaces it and the rest is searched anew. verify fails unless a progress file still replays with a game package.

Route memory

cache/reference/route.json lists, per 16 px cell of every beaten level, how many frames the level's route still needed from there to its exit, and is rewritten whenever a level's route changes. A row may end in what the run had there, in the game's dynamics() order (Snake Rattle 'n Roll: weight, pieces on the scale, speed); such a row counts only for a snake that has at least as much, so the map leads over nibbleys and power-ups too. The search weights cells on the route by the fewest frames left to the exit.

Reference run

On its first start the search downloads the video of the top verified human run on the game's speedrun.com leaderboard (speedrun.speedrun_com: game, category and variables) with yt-dlp to cache/reference/video.* and extracts 4 frames per second to cache/reference/frames/<seconds>s.png for the lab. It fails when that run has no video. Delete cache/reference/frames to fetch the run again.

The video is never read on its own: its frames are compared with screens the emulator rendered. At startup and every time the route memory is rewritten (after each beaten level and each improve round) one worker of the pool, unless the previous matching still runs, renders one frame after every placed cell of the search archives (cache/tas/archives, one save state per state and cell key) and caches the screen with its RAM position, level and dynamics() in cache/reference/screens.msgpack, so only new places are rendered. Video frames lose their black border (found from every tenth frame) and are cached in cache/reference/frames.npz. Both become 32x30 thumbnails. Rows that barely change across a level's emulator screens (HUD, borders: below a quarter of the median row spread) are left out, and the distance is 1 minus the Pearson correlation of the remaining pixels, so brightness and contrast of the capture do not count. The video splits into levels at dark score screens of at least 3 s, and a frame is compared only with the emulator screens of its level. It counts as matched only if its best distance is at most 0.2 and the best screen of a distinct place (another area or more than one 16 px cell away) is at least 0.1 further; every other frame stays unmatched, also when the level knows no second place. cache/reference/matches.json holds

{
  "levels": [{"level": 0, "frames": 640, "matched": 212,
              "waypoints": [{"seconds": 31.25, "cell": [3, 4, 9], "state": "Play"}]}],
  "matches": [{"seconds": 31.25, "state": "Play", "level": 0, "area": 3, "x": 70,
               "y": 150, "cell": [3, 4, 9], "distance": 0.08}]
}

where frames counts the level's frames that are not dark and the waypoints are the matched cells in video order, one per change of cell. cache/reference/video_route.json is a route memory file ([area, x // 16, y // 16, frame, frames_left, *dynamics]): per cell and dynamics the fewest emulator frames (at the emulator's frame rate) from the frame to the end of its level in the video, carrying what the human had there; the route map merges it with route.json. The stream gets metadata.speedrun.reference_progress, [{"level": 0, "matched": 212, "frames": 640}], and search.log a line per level.

Sprites and game knowledge

The engine reads what the NES draws, not what a picture looks like: every FCEUmm savestate holds the PPU's object memory (SPRA, 64 sprites of y, tile, attributes and x) and its control register (PPUR, bit 5 for 8x16 sprites). FCEUmm runs vertical blank and the game's sprite DMA before it draws a frame, so the object memory at the end of a frame is exactly the one on screen. No game setting is needed. Sprites whose 8x8 or 8x16 boxes touch or overlap form one object; an object is identified by its signature, the first 8 hex digits of a BLAKE2b hash of its sorted tile and palette pairs, so it keeps its signature wherever it moves and whichever way it faces.

Every 6 frames of each verified replay of the run and of the level reel, the objects on screen go into the sprite catalog cache/sprites/: <signature>.png is the object cut out of the screen when it was first seen, catalog.json maps each signature to when (UTC), in which level (RamInfo.level()) and state it was first seen and how often since. Every rendered video is drawn at twice the NES resolution with a thin box and a short label around each object: the player's name in yellow, a named sprite in cyan, ? in pink.

Names come from the game package's optional knowledge.yaml next to its config.yaml, which the lab writes for the website's "What the AI learned" page. A missing file is empty knowledge; a malformed one stops the run with the file and the field named. Every key is required:

mechanics: [{title: Weight, text: Each nibbley eaten makes the snake heavier.}]
player: {name: Rattle, text: The red snake., sprite: bda9d2fe}  # or null
enemies: [{name: Bigfoot, text: Stomps the snake flat., sprite: null, levels: [1, 2]}]
powerups: [{name: Clock, text: Stops the timer., sprite: null, effect: More time}]
levels: [{name: Level 1, goal: Ring the bell, route: Up the stairs, hazards: [water]}]
ram: [{address: 0x00A5, name: weight, meaning: Nibbleys eaten, verified: true}]

metadata.speedrun.knowledge publishes the file as it reads, and metadata.speedrun.sprites the 48 most seen catalog entries: [{signature, name, image, first_seen, first_level, first_state, seen}], with name from the knowledge or null and image the PNG as a data URI.

PPO policy

With speedrun.policy, a separate process trains a policy with Stable-Baselines3 PPO on the GPU, always from the start the search is working on (cache/tas/policy_start.msgpack). policy.env.PolicyEnv is a Gymnasium environment around one emulator: RAM in, a button combination out. Its reward is the state's reward, RAM values never seen on that level, a bonus for the exit and a penalty for a death; an episode ends on the exit, a death, another state or after 128 decisions. speedrun.policy_emulators environments play in parallel in a SubprocVecEnv. After every round of 1024 decisions per emulator the policy network (two tanh layers of 256) is exported to cache/tas/policy.npz, which the search workers read with numpy, the shortest episode that left the state is kept for the search to replay, and the round's reward, exits, entropy and value fit go to Weights & Biases.

Workflows and the lab

Flyte runs the work as tasks (datenwissenschaften.workflow): speedrun relays the stream, publishes the lab's status and searches and improves the movie forever. lab first starts the critic (agent/critic.sh) once five new reports landed in agent/reports/ since its last review; the stream shows its lessons as metadata.critic. Then it starts a lab run (agent/daily.sh of the game package) once no level stored a piece, the movie did not improve and no lab run ended for agent.stall_minutes minutes, as the journal records them; a checkpoint is no piece. Claude Code works for agent.minutes minutes on its own git worktree of the game package, and its branch ships only when its checks pass and the current movie still replays with it (speedrun.run verify). The lead session delegates to the subagents defined in the game package's .claude/agents/; the stream shows each member's role, state and latest step and every hand-off with its short task, the lead's latest actions (only the kind of action and the source files it touches), its commit subjects with URLs and secret-like words hidden, and the lines it changed. Every task runs locally with flyte run --local.

Experiments

The search and the policy each log one Weights & Biases run per release (project retro-speedlab, grouped by ui.release, jobs search and policy):

• search: the run from power-on as run/seconds, run/pieces, run/levels_beaten and the state it ends in, run/reached; every level's split as levels/<level>; running counts of search/found, search/improved, search/checkpoint, search/stuck and search/lab; the frames the trimming saved, search/frames_saved; and with every value search/hours_since_progress, the hours since a piece was found or made faster;
• policy: per PPO round policy/reward, policy/exits, policy/episodes, policy/exit_rate, policy/entropy and policy/explained_variance.

Without WANDB_API_KEY the runs are disabled and logged as not tracked.

Lab runs

After the verdict, python -m datenwissenschaften.lab.record CONFIG LOG STAMP BASE OUTCOME COMMITS (outcome rejected, notes or deployed) reads the run's Claude Code log once and

• publishes its numbers and a swimlane timeline (the lead and every subagent as lanes, tool calls as blocks, hand-offs as arrows) to the website at PUT /lab-runs/{game}/{stamp};
• sends it to Langfuse as the session lab-run STAMP, one trace per claude -p round, as OpenTelemetry spans with their recorded times. Every model turn is a generation with its token usage, every tool call a tool span and every subagent an agent span holding its own turns and tools. The session is scored with accepted, deployed, cost_usd, commits, minutes, turns, subagents and tool_errors.

Langfuse needs LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and LANGFUSE_BASE_URL; without them the run is logged as not traced.

Live relay

With ui.enable: true and an upload key, the engine relays to the Retro Speedlab backend (upload.url) every few seconds the snapshot (the published metadata and the release) and the search's newest frame; the whole verified movie goes up whenever it changes. The website's /stream page, opened with the stream key (?api_key=), plays them for OBS. The engine itself serves nothing.

Installation

Retro Speedlab Core requires Python 3.12.

git clone https://github.com/datenwissenschaften/retro-speedlab-core.git
cd retro-speedlab-core
uv sync
cp config.example.yaml config.yaml

The published package and import path are datenwissenschaften.

Minimal usage

This repository does not ship a playable game; a game package supplies the RAM layout, states, rewards and actions:

import numpy as np

from datenwissenschaften.environment.wrapper import StateMachineGymWrapper


class AirstrikerWrapper(StateMachineGymWrapper[AirstrikerRam]):
    start_state_cls = SurviveAndScore
    state_classes = (SurviveAndScore,)
    ram_info_cls = AirstrikerRam
    action_table = np.array([[[1] + [0] * 11, [0] * 12, [0] * 12, [0] * 12]], dtype=np.int8)
    levels = ()

python -m datenwissenschaften.speedrun.run search airstriker:AirstrikerWrapper config.yaml SurviveAndScore

Every state sets a one-line goal. The ROM is imported through Stable Retro from paths.roms; the engine's own test suite needs no ROM.

Configuration

Every key in config.example.yaml is required; missing or malformed values raise an error when the configuration is loaded.

Testing

uv run pytest --cov=datenwissenschaften --cov-fail-under=95

Tests run on a scripted emulator, so they need no ROM.

Development

uv run ruff check .
uv run ruff format --check .

Relationship to Retro Speedlab

• Retro Speedlab Core (this repository): the engine: segment search, verified movies, the live relay and the lab workflow.
• Game packages: one repository per game with its RAM map, states, rewards, actions and Dokku deployment, improved by lab runs.
• retrospeedlab.com: the lab's website, showing the stream and the speedrun.

License

Copyright © datenwissenschaften contributors.

Distributed under the GNU General Public License v3.0.