Simple efficient data recording for OpenAI Gym reinforcement learning environments
Project description
Gymbag is a Python 3 library for easy, efficient, single-file storage of OpenAI Gym reinforcement learning environment data. It stores observations, actions, and rewards in portable, compressed HDF5 files. You can easily playback the data for training or testing, or read it in for analysis.
Gymbag automatically appends to existing files, so you can easily accumulate all your training data in one place. You can also store multiple data sets from separate experiments or environments in the same file, along with descriptions to keep them straight. You can store arbitrary metadata with each step (in the info dict). Gymbag does not store rendered output (movies). It’s also easy to add other storage formats.
Recording
# Wrap env with HDF5 recorder, specifying filename
env = record_hdf5(gym.make('CartPole-v0'), 'cartpole.h5')
for episode in range(2):
env.reset()
done = False
while not done:
obs, reward, done, info = env.step(env.action_space.sample())
Reading
Reading the data back is just as easy:
for episode in HDF5Reader('cartpole.h5'):
for time, obs, action, reward, done, info in episode:
print(time, obs, action, reward, done, info)
1500617412.784789 [ 0.03698143 0.01418633 0.01207788 0.00391994] 0.0 nan False None
1500617412.78494 [ 0.03726516 -0.18110673 0.01215627 0.30038899] 0.0 1.0 False None
1500617412.784968 [ 0.03364302 0.01383986 0.01816405 0.01156456] 1.0 1.0 False None
1500617412.784994 [ 0.03391982 0.20869666 0.01839535 -0.27533251] 1.0 1.0 False None
1500617412.785022 [ 0.03809375 0.01331716 0.0128887 0.02309511] 0.0 1.0 False None
...
Playback
env = PlaybackEnv(HDF5Reader('cartpole.h5'))
while not env.played_out:
env.reset()
done = False
while not done:
obs, reward, done, info = env.step(env.action_space.sample())
print(obs, reward, done, info)
Performance
Typically wrapping an environment with Gymbag results in less than a 2X slowdown. Here is a comparison to gym_recording (which dumps data as uncompressed binary blobs):
Environment |
Time (s) |
Space (MB) |
|||
---|---|---|---|---|---|
unwrapped |
gym_recording |
gymbag.hdf5 |
gym_recording |
gymbag.hdf5 |
|
Cartpole-v0 |
0.38 (1X) |
0.63 (1.7X) |
0.64 (1.7X) |
1.6 (3.2X) |
0.5 (1X) |
Breakout-v0 |
1.72 (1X) |
2.20 (1.3X) |
3.34 (1.9X) |
223 (74X) |
3 (1X) |
Extras
Also comes with tools for recording to memory, generating test data, comparing data between runs, converting saved data formats, and more.
Documentation
Gymbag is open-source licensed under the LGPL 3.0.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.