Skip to main content

Simple efficient data recording for OpenAI Gym reinforcement learning environments

Project description

Gymbag is a Python 3 library for easy, efficient, single-file storage of OpenAI Gym reinforcement learning environment data. It stores observations, actions, and rewards in portable, compressed HDF5 files. You can easily playback the data for training or testing, or read it in for analysis.

Gymbag automatically appends to existing files, so you can easily accumulate all your training data in one place. You can also store multiple data sets from separate experiments or environments in the same file, along with descriptions to keep them straight. You can store arbitrary metadata with each step (in the info dict). Gymbag does not store rendered output (movies). It’s also easy to add other storage formats.

https://gitlab.com/doctorj/gymbag/badges/master/build.svg

Recording

# Wrap env with HDF5 recorder, specifying filename
env = record_hdf5(gym.make('CartPole-v0'), 'cartpole.h5')
for episode in range(2):
    env.reset()
    done = False
    while not done:
        obs, reward, done, info = env.step(env.action_space.sample())

Reading

Reading the data back is just as easy:

for episode in HDF5Reader('cartpole.h5'):
    for time, obs, action, reward, done, info in episode:
        print(time, obs, action, reward, done, info)
1500617412.784789 [ 0.03698143  0.01418633  0.01207788  0.00391994] 0.0 nan False None
1500617412.78494 [ 0.03726516 -0.18110673  0.01215627  0.30038899] 0.0 1.0 False None
1500617412.784968 [ 0.03364302  0.01383986  0.01816405  0.01156456] 1.0 1.0 False None
1500617412.784994 [ 0.03391982  0.20869666  0.01839535 -0.27533251] 1.0 1.0 False None
1500617412.785022 [ 0.03809375  0.01331716  0.0128887   0.02309511] 0.0 1.0 False None
...

Playback

env = PlaybackEnv(HDF5Reader('cartpole.h5'))

while not env.played_out:
    env.reset()
    done = False
    while not done:
        obs, reward, done, info = env.step(env.action_space.sample())
        print(obs, reward, done, info)

Performance

Typically wrapping an environment with Gymbag results in less than a 2X slowdown. Here is a comparison to gym_recording (which dumps data as uncompressed binary blobs):

Environment Time (s) Space (MB)
  unwrapped gym_recording gymbag.hdf5 gym_recording gymbag.hdf5
Cartpole-v0 0.38 (1X) 0.63 (1.7X) 0.64 (1.7X) 1.6 (3.2X) 0.5 (1X)
Breakout-v0 1.72 (1X) 2.20 (1.3X) 3.34 (1.9X) 223 (74X) 3 (1X)

Extras

Also comes with tools for recording to memory, generating test data, comparing data between runs, converting saved data formats, and more.

Documentation

Gymbag is open-source licensed under the LGPL 3.0.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for gymbag, version 0.4.9
Filename, size File type Python version Upload date Hashes
Filename, size gymbag-0.4.9.tar.gz (12.7 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page