replay-monitor 0.0.3

A tool for easy data exploration in reinforcement learning environments.

#Replay Monitor

This is a tool for recording and observing data and measurements generated through the interactions between a reinforcement learning algorithm and an environment with an OpenAI Gym interface.

Currently, this tool offers two main features:

• A convenient environment wrapper that allows the user to:

  • Record Tensorboard metrics during the training of the RL agent
  • Log the entire interaction with the environment in a local DB (for later use with the interactive tool below).

• An interactive tool that visualize stored interactions (episodes and transitions) on-demand.

This tool supports complex state spaces, including tuple spaces.

Note: This is a premature release, keep in mind that since this package is still in development, bugs and changes are expected.

Installation

Install the package by

pip install replay-monitor

Usage Examples

Record Agent Interactions

To use the environment wrapper for storing interactions:

from replay_monitor import Monitor
import gym

env = gym.make('Breakout-v0')
env = Monitor(env, log_to_db=True)

Now, you can use the environment as usual, for example:

env.reset()
for i in range(300):
    action = env.action_space.sample()
    state, reward, done, info = env.step(action)

    if done:
        env.reset()
...

Use The Interactive Tool

Run the interactive tool by executing the following command in the command-line (make sure your environment is activated if you use virtualenv):

replay-monitor --db_path <db_path>

where <db_path> is the path to the .h5 file generated by the environment wrapper Monitor (you can omit --db_path if you use the default value).

Record Tensorboard Metrics

TODO