gymbag 0.4.9

Simple efficient data recording for OpenAI Gym reinforcement learning environments

Gymbag is a Python 3 library for easy, efficient, single-file storage of OpenAI Gym reinforcement learning environment data. It stores observations, actions, and rewards in portable, compressed HDF5 files. You can easily playback the data for training or testing, or read it in for analysis.

Gymbag automatically appends to existing files, so you can easily accumulate all your training data in one place. You can also store multiple data sets from separate experiments or environments in the same file, along with descriptions to keep them straight. You can store arbitrary metadata with each step (in the info dict). Gymbag does not store rendered output (movies). It’s also easy to add other storage formats.

Recording

# Wrap env with HDF5 recorder, specifying filename
env = record_hdf5(gym.make('CartPole-v0'), 'cartpole.h5')
for episode in range(2):
    env.reset()
    done = False
    while not done:
        obs, reward, done, info = env.step(env.action_space.sample())

Reading

Reading the data back is just as easy:

for episode in HDF5Reader('cartpole.h5'):
    for time, obs, action, reward, done, info in episode:
        print(time, obs, action, reward, done, info)

1500617412.784789 [ 0.03698143  0.01418633  0.01207788  0.00391994] 0.0 nan False None
1500617412.78494 [ 0.03726516 -0.18110673  0.01215627  0.30038899] 0.0 1.0 False None
1500617412.784968 [ 0.03364302  0.01383986  0.01816405  0.01156456] 1.0 1.0 False None
1500617412.784994 [ 0.03391982  0.20869666  0.01839535 -0.27533251] 1.0 1.0 False None
1500617412.785022 [ 0.03809375  0.01331716  0.0128887   0.02309511] 0.0 1.0 False None
...

Playback

env = PlaybackEnv(HDF5Reader('cartpole.h5'))

while not env.played_out:
    env.reset()
    done = False
    while not done:
        obs, reward, done, info = env.step(env.action_space.sample())
        print(obs, reward, done, info)

Performance

Typically wrapping an environment with Gymbag results in less than a 2X slowdown. Here is a comparison to gym_recording (which dumps data as uncompressed binary blobs):

Environment	Time (s)	Space (MB)
	unwrapped	gym_recording	gymbag.hdf5	gym_recording	gymbag.hdf5
Cartpole-v0	0.38 (1X)	0.63 (1.7X)	0.64 (1.7X)	1.6 (3.2X)	0.5 (1X)
Breakout-v0	1.72 (1X)	2.20 (1.3X)	3.34 (1.9X)	223 (74X)	3 (1X)

Extras

Also comes with tools for recording to memory, generating test data, comparing data between runs, converting saved data formats, and more.

Documentation

• GitLab Repo

• Documentation: Examples | API

Gymbag is open-source licensed under the LGPL 3.0.