Skip to main content

PyTorch Reinforcement Learning Framework for Researchers

Project description

Test Status

Cherry is a reinforcement learning framework for researchers built on top of PyTorch.

Unlike other reinforcement learning implementations, cherry doesn't try to provide a single interface to existing algorithms. Instead, it provides you with common tools to write your own algorithms. Drawing from the UNIX philosophy, each tool strives to be as independent from the rest of the framework as possible. So if you don't like a specific tool, you can still use parts of cherry without headaches.


  • Pythonic and modular interface à la Pytorch.
  • Support for tabular (!) and function approximation algorithms.
  • Various OpenAI Gym environment wrappers.
  • Helper functions for popular algorithms. (e.g. A2C, DDPG, TRPO, PPO, SAC)
  • Logging, visualization, and debugging tools.
  • Painless and efficient distributed training on CPUs and GPUs.
  • Unit, integration, and regression tested, continuously integrated.


For now, cherry is still in development.

  1. Clone the repo: git clone
  2. cd cherry
  3. pip install -e .

Upon our first public release, you'll be able to

pip install cherry-rl

Development Guidelines

  • The master branch is always working, considered stable.
  • The dev branch should always work and is ahead of master, considered cutting edge.
  • To implement a new functionality: branch dev into your_name/functionality_name, implement your functionality, then pull request to dev. It will be periodically merged into master.


The following snippet demonstrates some of the tools offered by cherry.

import cherry as ch

# Wrapping environments
env = ch.envs.Logger(env, interval=1000)  # Prints rollouts statistics
env = ch.envs.Normalized(env, normalize_state=True, normalize_reward=False)  
env = ch.envs.Torch(env)  # Converts actions/states to tensors

# Storing and retrieving experience
replay = ch.ExperienceReplay()
replay.append(old_state, action, reward, state, done, info = {
    'log_prob': mass.log_prob(action),  # Can add any variable/tensor to the transitions
    'value': value
replay.actions  # Tensor of all stored actions
replay.states  # Tensor of all stored states
replay.empty()  # Removes all stored experience

# Discounting and normalizing rewards
rewards =, replay.rewards, replay.dones)
rewards = ch.utils.normalize(rewards)

# Sampling rollouts per episode or samples
num_samples, num_episodes = ch.rollouts.collect(env,
                                                # alternatively: num_samples=1000,

Concrete examples are available in the examples/ folder.


The documentation will be written as we begin to converge the core concepts of cherry.


Some functionalities that we might want to implement.

  • parallelize environments and a way to handle it with ExperienceReplay,
  • VisdomLogger as a dashboard to debug an implementation,
  • example with reccurent net,
  • minimal but complete documentation,
  • GPU implementations.


Cherry draws inspiration from many reinforcement learning implementations, including

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for cherry-rl, version 0.0.7
Filename, size File type Python version Upload date Hashes
Filename, size cherry-rl-0.0.7.tar.gz (26.0 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page