raylab 0.7.0

Reinforcement learning algorithms in RLlib and PyTorch.

raylab

Reinforcement learning algorithms in RLlib and PyTorch.

Introduction

Raylab provides agents and environments to be used with a normal RLlib/Tune setup.

import ray
from ray import tune
import raylab

def main():
    raylab.register_all_agents()
    raylab.register_all_environments()
    ray.init()
    tune.run(
        "NAF",
        local_dir=...,
        stop={"timesteps_total": 100000},
        config={
            "env": "CartPoleSwingUp-v0",
            "exploration_config": {
                "type": tune.grid_search([
                    "raylab.utils.exploration.GaussianNoise",
                    "raylab.utils.exploration.ParameterNoise"
                ])
            }
            ...
        },
    )

if __name__ == "__main__":
    main()

One can then visualize the results using raylab dashboard

Installation

pip install raylab

Algorithms

Paper	Agent Name
Actor Critic using Kronecker-factored Trust Region	ACKTR
Trust Region Policy Optimization	TRPO
Normalized Advantage Function	NAF
Stochastic Value Gradients	SVG(inf)/SVG(1)/SoftSVG
Soft Actor-Critic	SoftAC
Model-Based Policy Optimization	MBPO
Streamlined Off-Policy (DDPG)	SOP

Command-line interface

For a high-level description of the available utilities, run raylab –help

Usage: raylab [OPTIONS] COMMAND [ARGS]...

RayLab: Reinforcement learning algorithms in RLlib.

Options:
  --help  Show this message and exit.

  Commands:
    dashboard    Launch the experiment dashboard to monitor training progress.
    experiment   Launch a Tune experiment from a config file.
    find-best    Find the best experiment checkpoint as measured by a metric.
    plot         Draw lineplots of the relevant variables and display them on...
    plot-export  Draw lineplots of the relevant variables and save them as...
    rollout      Simulate an agent from a given checkpoint in the desired...

Packages

The project is structured as follows

raylab
├── agents            # Trainer and Policy classes
├── cli               # Command line utilities
├── distributions     # Extendend and additional PyTorch distributions
├── envs              # Gym environments
├── logger            # Tune loggers
├── modules           # PyTorch neural network modules for algorithms
    ├── basic         # Building blocks for neural networks
    ├── flows         # Normalizing Flow modules
    ├── distributions # TorchScript compatible distribution modules
├── policy            # Extensions and customizations of RLlib's policy API
├── utils             # miscellaneous utilities

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

History

0.6.5 (2020-05-21)

• First release on PyPI.