Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Reinforcement Learning in Pytorch

Project description

Deep Reinforcement Learning in Pytorch

This repository contains all standard model-free and model-based(coming) RL algorithms in Pytorch. (May also contain some research ideas I am working on currently)

What is it?

pytorch-rl implements some state-of-the art deep reinforcement learning algorithms in Pytorch, especially those concerned with continuous action spaces. You can train your algorithm efficiently either on CPU or GPU. Furthermore, pytorch-rl works with OpenAI Gym out of the box. This means that evaluating and playing around with different algorithms is easy. Of course you can extend pytorch-rl according to your own needs. TL:DR : pytorch-rl makes it really easy to run state-of-the-art deep reinforcement learning algorithms.

Dependencies

  1. Pytorch
  2. Gym (OpenAI)
  3. mujoco-py (For the physics simulation and the robotics env in gym)
  4. Pybullet (Coming Soon)
  5. MPI (Only supported with mpi backend Pytorch installation)
  6. Tensorboardx (https://github.com/lanpa/tensorboardX)

RL algorithms

  1. DQN (with Double Q learning)
  2. DDPG
  3. DDPG with HER (For the OpenAI Fetch Environments)
  4. Heirarchical Reinforcement Learning
  5. Prioritized Experience Replay + DDPG
  6. DDPG with Prioritized Hindsight experience replay (Research)
  7. Neural Map with A3C (Coming Soon)
  8. Rainbow DQN (Coming Soon)
  9. PPO
  10. HER with self attention for goal substitution (Research)
  11. A3C (Coming Soon)
  12. ACER (Coming Soon)
  13. DARLA
  14. TDM
  15. World Models
  16. Soft Actor-Critic

Environments

  1. Breakout
  2. Pong (coming soon)
  3. Hand Manipulation Robotic Task
  4. Fetch-Reach Robotic Task
  5. Hand-Reach Robotic Task
  6. Block Manipulation Robotic Task
  7. Montezuma's Revenge (Current Research)
  8. Pitfall
  9. Gravitar
  10. CarRacing
  11. OpenSim Prosthetics Nips Challenge (https://www.crowdai.org/challenges/nips-2018-ai-for-prosthetics-challenge)

Environment Modelling (For exploration and domain adaptation)

Multiple GAN training tricks have been used because of the instability in training the generators and discriminators. Please refer to https://github.com/soumith/ganhacks for more information.

Even after using the tricks, it was really hard to train a GAN to convergence. However, after using Spectral Normalization (https://arxiv.org/abs/1802.05957) the infogan was trained to convergence.

For image to image translation tasks with GANs and for VAEs in general, training with Skip Connection really helps the training.

  1. beta-VAE
  2. InfoGAN
  3. CVAE-GAN
  4. Flow based generative models (Research)
  5. SAGAN
  6. Sequential Attend, Infer, Repeat
  7. Curiosity driven exploration
  8. Parameter Space Noise for Exploration
  9. Noisy Network

References

  1. Playing Atari with Deep Reinforcement Learning, Mnih et al., 2013
  2. Human-level control through deep reinforcement learning, Mnih et al., 2015
  3. Deep Reinforcement Learning with Double Q-learning, van Hasselt et al., 2015
  4. Continuous control with deep reinforcement learning, Lillicrap et al., 2015
  5. CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training, Bao et al., 2017
  6. beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework, Higgins et al., 2017
  7. Hindsight Experience Replay, Andrychowicz et al., 2017
  8. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets, Chen et al., 2016
  9. World Models, Ha et al., 2018
  10. Spectral Normalization for Generative Adversarial Networks, Miyato et al., 2018
  11. Self-Attention Generative Adversarial Networks, Zhang et al., 2018
  12. Curiosity-driven Exploration by Self-supervised Prediction, Pathak et al., 2017
  13. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor, Haarnoja et al., 2018
  14. Parameter Space Noise for Exploration, Plappert et al., 2018
  15. Noisy Network for Exploration, Fortunato et al., 2018
  16. Proximal Policy Optimization Algorithms, Schulman et al., 2017

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pytorch-policy, version 0.1.1
Filename, size File type Python version Upload date Hashes
Filename, size pytorch_policy-0.1.1-py3-none-any.whl (3.1 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size pytorch-policy-0.1.1.tar.gz (55.4 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page