Procgen Gymnasium

A Gymnasium port of OpenAI's Procgen Benchmark — 16 procedurally-generated environments for Reinforcement Learning (RL) research.

[Original Paper] [Original Blog Post]

What is this?

This project modernises the original procgen package (which depends on the deprecated gym and unmaintained gym3 libraries) to work with Gymnasium (the maintained successor to OpenAI Gym) and Python 3.13+.

The C++ game code is unchanged — only the Python interface layer has been rewritten.

Key changes from the original

• gym / gym3 replaced with gymnasium
• Supports gymnasium.make() (single env) and gymnasium.make_vec() (vectorized)
• Compatible with all standard Gymnasium wrappers
• Python 3.13+ support
• Managed with uv and pyproject.toml

Installation

pip install procgen-gym

Requirements: Python 3.13+ (64-bit).

Platform	Architecture	Status
Linux	x86_64	Supported
Windows	AMD64	Supported
macOS	-	Not supported

Quick Start

Single environment (standard Gymnasium API)

import gymnasium as gym
import procgen_gym  # registers environments

env = gym.make("procgen_gym/procgen-coinrun-v0")
obs, info = env.reset()

for _ in range(1000):
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        obs, info = env.reset()

env.close()

Vectorized environment (native batched)

import gymnasium as gym
import numpy as np
import procgen_gym  # registers environments

env = gym.make_vec("procgen_gym/procgen-coinrun-v0", num_envs=16)
obs, info = env.reset()

for _ in range(1000):
    actions = np.array([env.action_space.sample() for _ in range(16)])
    obs, reward, terminated, truncated, info = env.step(actions)

env.close()

Direct instantiation

from procgen_gym import ProcgenEnv, ProcgenVecEnv

# Single environment
env = ProcgenEnv(env_name="coinrun")

# Vectorized environment (C++ level batching)
vec_env = ProcgenVecEnv(num_envs=16, env_name="coinrun")

Environments

All environments produce (64, 64, 3) RGB observations and use a Discrete(15) action space. See docs/environments/ for detailed per-environment documentation.

Screenshot	Name	Description
	bigfish	Eat smaller fish to grow, avoid larger fish
	bossfight	Dodge projectiles and destroy a boss starship
	caveflyer	Navigate cave networks in an Asteroids-style ship
	chaser	Collect orbs in a maze while avoiding enemies (Pac-Man inspired)
	climber	Climb platforms and collect stars while dodging enemies
	coinrun	Platformer — reach the coin at the far right
	dodgeball	Navigate rooms and throw balls at enemies (Berzerk inspired)
	fruitbot	Collect fruit, avoid non-fruit objects in a scrolling game
	heist	Collect coloured keys to unlock doors and steal the gem
	jumper	Open-world platformer with double-jump to find the carrot
	leaper	Cross lanes of traffic and hop across river logs (Frogger inspired)
	maze	Navigate a procedurally-generated maze to find cheese
	miner	Dig through dirt, avoid falling boulders, collect diamonds (Boulder Dash inspired)
	ninja	Precision platformer with charged jumps and throwing stars
	plunder	Naval cannon combat — destroy enemies, spare friendlies
	starpilot	Side-scrolling space shooter with targeting enemies

Environment Options

All options can be passed as keyword arguments to gym.make(), gym.make_vec(), or the ProcgenVecEnv constructor:

Option	Default	Description
num_levels	0	Number of unique levels (0 = unlimited)
start_level	0	Lowest seed used to generate levels
distribution_mode	"hard"	Level variant: "easy", "hard", "extreme", "memory", "exploration"
paint_vel_info	False	Paint player velocity info in top left corner
use_generated_assets	False	Use randomly generated assets instead of designed ones
use_backgrounds	True	Use designed backgrounds (False for pure black)
restrict_themes	False	Restrict to single asset theme
use_monochrome_assets	False	Use monochromatic rectangles instead of designed assets
center_agent	True	Center observations on the agent
use_sequential_levels	False	Don't end episode at level completion, continue to next
num_threads	4	Number of C++ threads for environment stepping
rand_seed	Random	Random seed for level generation
debug_mode	0	Debug flag passed through to C++ game code

Gymnasium Wrapper Compatibility

ProcgenVecEnv and ProcgenEnv are compatible with standard Gymnasium wrappers.

Vector wrappers (gymnasium.wrappers.vector)

Wrapper	Compatible	Notes
GrayscaleObservation	Yes	RGB to grayscale conversion
ResizeObservation	Yes	Resize 64x64 to any resolution
ReshapeObservation	Yes	Reshape observation arrays
RescaleObservation	Yes	Best used after DtypeObservation(float)
DtypeObservation	Yes	Convert uint8 to float32/float64
FlattenObservation	Yes	Flatten to 1D (12288,)
NormalizeObservation	Yes	Running mean/std normalisation
TransformObservation	Yes	Custom observation function
ClipReward	Yes	Bound rewards to a range
NormalizeReward	Yes	Normalise rewards via running stats
TransformReward	Yes	Custom reward function
TransformAction	Yes	Custom action function
RecordEpisodeStatistics	Yes	Track episode returns and lengths
RecordVideo	Yes	Requires render_mode="rgb_array" and moviepy
HumanRendering	Yes	Requires render_mode="rgb_array", pygame, and opencv
DictInfoToList	Yes	Convert info dict to list of dicts
ClipAction	No	Discrete action space (not Box)
RescaleAction	No	Discrete action space (not Box)
FilterObservation	No	Box observation space (not Dict)

Single-env wrappers (gymnasium.wrappers)

Wrapper	Compatible	Notes
FrameStackObservation	Yes	Stack N recent frames
DelayObservation	Yes	Add observation delay
TimeAwareObservation	Yes	Append time step to observation
TimeLimit	Yes	Truncate after N steps
MaxAndSkipObservation	Yes	Frame skipping with max pooling
StickyAction	Yes	Probabilistic action repeat
GrayscaleObservation	Yes	RGB to grayscale
ResizeObservation	Yes	Resize observations
RecordEpisodeStatistics	Yes	Episode tracking
RenderCollection	Yes	Collect rendered frames
AddRenderObservation	Yes	Add rendered frame to obs dict

State Save/Load

from procgen_gym import ProcgenVecEnv

env = ProcgenVecEnv(num_envs=1, env_name="coinrun")
env.reset()

# Save state
states = env.get_state()

# ... take some actions ...

# Restore state
env.set_state(states)

Interactive Play

procgen-interactive --env-name coinrun

Keys: arrow keys + Q, W, E, A, S, D for actions. Score is displayed on screen.

Building from Source

git clone https://github.com/Achronus/procgen-gymnasium.git
cd procgen-gymnasium
uv sync --extra test
uv run python -c "from procgen_gym import ProcgenVecEnv; env = ProcgenVecEnv(num_envs=1, env_name='coinrun'); print('OK'); env.close()"

The C++ code requires Qt5 for rendering. On Windows, install via vcpkg:

vcpkg install qt5-base:x64-windows
set PROCGEN_CMAKE_PREFIX_PATH=path/to/vcpkg_installed/x64-windows/share/cmake

Building Wheels

Requires MSVC, CMake, vcpkg with Qt5, and Docker Desktop (for Linux wheels).

# Install build dependencies
uv sync --extra build

# Build both Windows + Linux wheels
uv run python -m procgen_build --clean

# Or one platform at a time
uv run python -m procgen_build --platform win
uv run python -m procgen_build --platform linux

# Verify Windows wheel has Qt5 DLLs bundled (should show 10+ DLLs)
uv run python -c "import zipfile, glob; whl=glob.glob('wheelhouse/*win*.whl')[0]; dlls=[f for f in zipfile.ZipFile(whl).namelist() if f.endswith('.dll')]; print(f'{len(dlls)} DLLs:'); [print(f'  {d}') for d in dlls]"

# Verify Linux wheel has libenv.so with Qt5 bundled (should show ~18MB .so)
uv run python -c "import zipfile, glob; whl=glob.glob('wheelhouse/*manylinux*.whl')[0]; libs=[f for f in zipfile.ZipFile(whl).namelist() if f.endswith('.so')]; print(f'{len(libs)} .so libs:'); [print(f'  {f}') for f in libs]"

Known Issues

These are inherited from the original procgen and kept for reproducibility:

• bigfish — Player can occasionally become trapped along environment borders
• caveflyer — ~0.5% of levels spawn the player next to an enemy (instant death)
• jumper — ~7% of levels spawn the player on top of an enemy or goal (instant termination)
• miner — Low probability of unsolvable configurations

Citation

@article{cobbe2019procgen,
  title={Leveraging Procedural Generation to Benchmark Reinforcement Learning},
  author={Cobbe, Karl and Hesse, Christopher and Hilton, Jacob and Schulman, John},
  journal={arXiv preprint arXiv:1912.01588},
  year={2019}
}