An easy-to-use reinforcement learning library for research and education
Project description
A Reinforcement Learning Library for Research and Education
Section | Description |
---|---|
Introduction | The philosophy of rlberry |
Citation | How to cite this work |
Getting started | A quick usage guide of rlberry |
Installation | How to install rlberry |
Documentation | A link to the documentation |
Contributing | A guide for contributing |
Introduction
The goal of rlberry
to make Reinforcement Learning (RL) research and teaching easier. For that purpose, rlberry
provides several interfaces that allow users to create their own experimental pipeline in a fast, clear and reproducible way. rlberry
comprises the following major components and main features:
-
An interface for agents, that
-
puts minimal constraints on the agent code (=> making it easy to include new algorithms and modify existing ones);
-
allows comparison between agents using a simple and unified evaluation interface (=> making it easy, for instance, to compare deep and "traditional" RL algorithms);
-
includes detailed documentation and comprehensible tutorial/examples (Jupyter Notebook) for each implemented agents (=> making it easy for education).
-
-
An interface for environments, that
-
allows to create novel environments easily (=> making it possible to add your own environments);
-
totally adapts to
OpenAI Gym
(=> making it easy to use any existing environments fromgym
).
-
-
An interface for rendering, that
- provides simple and clear visualization of your experiments.
-
Several important features include
-
a unified seeding mechanism: define only one global seed, from which all other seeds will inherit, enforcing independence of the random number generators;
-
an interface to
Optuna
that allows automatic hyperparameter optimization; -
compatibility with
Sacred
that facilitates configuration, organization, logging and reproducing of computational experiments.
-
Citing rlberry
If you use rlberry
in scientific publications, we would appreciate citations using the following Bibtex entry:
@misc{rlberry,
author = {Domingues, Omar Darwiche and Flet-Berliac, Yannis and Leurent, Edouard and M{\'e}nard, Pierre and Shang, Xuedong and Valko, Michal},
title = {{rlberry - A Reinforcement Learning Library for Research and Education}},
year = {2021},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/rlberry-py/rlberry}},
}
Getting started
We provide a handful of notebooks on Google colab as examples to show you how to use rlberry
.
Compatibility with OpenAI Gym
If you want to use gym
environments with rlberry
, simply do the following:
from rlberry.envs import gym_make
# for example, let's take CartPole
env = gym_make('CartPole-v1')
This way, env
behaves exactly the same as the gym
environment, we simply replace the seeding function by env.reseed()
, which ensures unified seeding and reproducibility when using rlberry
.
Seeding
In rlberry
, only one global seed is defined, and all the random number generators used by the agents and environments inherit from this seed, ensuring reproducibility and independence between the generators (see NumPy SeedSequence).
Example:
import rlberry.seeding as seeding
seeding.set_global_seed(seed=123)
# From now on, no more seeds are defined by the user, and all the results are reproducible.
...
# If you need a random number generator (rng), call:
rng = seeding.get_rng()
# which gives a numpy Generator (https://numpy.org/doc/stable/reference/random/generator.html)
# that is independent of all the previous generators created by seeding.get_rng()
rng.integers(5)
rng.normal()
# etc
Installation
Cloning & creating virtual environment
It is suggested to create a virtual environment using Anaconda or Miniconda:
git clone https://github.com/rlberry-py/rlberry.git
conda create -n rlberry python=3.7
Basic installation
Install without heavy libraries (e.g. pytorch):
conda activate rlberry
pip install -e .
Full installation
Install with all features:
conda activate rlberry
pip install -e .[full]
which includes:
Numba
for just-in-time compilation of algorithms based on dynamic programmingPyTorch
for Deep RL agentsOptuna
for hyperparameter optimizationSacred
for handling experiment configurationsffmpeg-python
for saving videosPyOpenGL
for more rendering options
Tests
To run tests, install test dependencies with pip install -e .[test]
and run pytest
. To run tests with coverage, install test dependencies and run bash run_testscov.sh
. See coverage report in cov_html/index.html
.
Documentation
The documentation is under construction and will be available here.
Contributing
Want to contribute to rlberry
? Please check our contribution guidelines. A list of interesting TODO's will be available soon. If you want to add any new agents or environments, do not hesitate to open an issue!
Implementation notes
- When inheriting from the
Agent
class, make sure to callAgent.__init__(self, env, **kwargs)
using**kwargs
in case new features are added to the base class, and to make sure thatcopy_env
andreseed_env
are always an option to any agent.
Infos, errors and warnings are printed using the logging
library.
- From
gym
torlberry
:reseed
(rlberry) should be called instead ofseed
(gym).seed
keeps compatilibity with gym, whereasreseed
uses the unified seeding mechanism ofrlberry
.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.