NeodroidAgent 0.4.4

Reinforcement learning agent implementations, intended for use with the Neodroid platform

Agent

This repository will host all initial machine learning efforts applying the Neodroid platform.

---

Neodroid is developed with support from Research Council of Norway Grant #262900. (https://www.forskningsradet.no/prosjektbanken/#/project/NFR/262900)

---

Contents Of This Readme

• Algorithms
• Requirements
• Usage
• Results
  • Target Point Estimator
  • Perfect Information Navigator
• Contributing
• Other Components

Algorithms

• REINFORCE (PG)
• DQN
• DDPG
• PPO
• TRPO, GA, EVO, IMITATION...

Algorithms Implemented

1. Deep Q Learning (DQN) _{^{(Mnih et al. 2013)}}
2. DQN with Fixed Q Targets _{^{(Mnih et al. 2013)}}
3. Double DQN (DDQN) _{^{(Hado van Hasselt et al. 2015)}}
4. DDQN with Prioritised Experience Replay _{^{(Schaul et al. 2016)}}
5. Dueling DDQN _{^{(Wang et al. 2016)}}
6. REINFORCE _{^{(Williams et al. 1992)}}
7. Deep Deterministic Policy Gradients (DDPG) _{^{(Lillicrap et al. 2016 )}}
8. Twin Delayed Deep Deterministic Policy Gradients (TD3) _{^{(Fujimoto et al. 2018)}}
9. Soft Actor-Critic (SAC & SAC-Discrete) _{^{(Haarnoja et al. 2018)}}
10. Asynchronous Advantage Actor Critic (A3C) _{^{(Mnih et al. 2016)}}
11. Syncrhonous Advantage Actor Critic (A2C)
12. Proximal Policy Optimisation (PPO) _{^{(Schulman et al. 2017)}}
13. DQN with Hindsight Experience Replay (DQN-HER) _{^{(Andrychowicz et al. 2018)}}
14. DDPG with Hindsight Experience Replay (DDPG-HER) _{^{(Andrychowicz et al. 2018 )}}
15. Hierarchical-DQN (h-DQN) _{^{(Kulkarni et al. 2016)}}
16. Stochastic NNs for Hierarchical Reinforcement Learning (SNN-HRL) _{^{(Florensa et al. 2017)}}
17. Diversity Is All You Need (DIAYN) _{^{(Eyensbach et al. 2018)}}

Environments Implemented

1. Bit Flipping Game _{^{(as described in Andrychowicz et al. 2018)}}
2. Four Rooms Game _{^{(as described in Sutton et al. 1998)}}
3. Long Corridor Game _{^{(as described in Kulkarni et al. 2016)}}
4. Ant-{Maze, Push, Fall} _{^{(as desribed in Nachum et al. 2018 and their accompanying code)}}

Requirements

• pytorch
• tqdm
• Pillow
• numpy
• matplotlib
• torchvision
• torch
• Neodroid
• pynput

(Optional)

• visdom
• gym

To install these use the command:

pip3 install -r requirements.txt

Usage

Export python path to the repo root so we can use the utilities module

export PYTHONPATH=/path-to-repo/

For training a agent use:

python3 procedures/train_agent.py

For testing a trained agent use:

python3 procedures/test_agent.py

Results

Target Point Estimator

Using Depth, Segmentation And RGB images to estimate the location of target point in an environment.

REINFORCE (PG)

DQN

DDPG

PPO

GA, EVO, IMITATION...

Perfect Information Navigator

Has access to perfect location information about the obstructions and target in the environment, the objective is to navigate to the target with colliding with the obstructions.

REINFORCE (PG)

DQN

DDPG

PPO

GA, EVO, IMITATION...

Contributing

See guidelines for contributing here.

Licensing

This project is licensed under the Apache V2 License. See LICENSE for more information.

Citation

For citation you may use the following bibtex entry:

@misc{neodroid-agent,
  author = {Heider, Christian},
  title = {Neodroid Platform Agents},
  year = {2018},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/sintefneodroid/agent}},
}

Other Components Of the Neodroid Platform

• neo
• droid

Authors

• Christian Heider Nielsen - cnheider

Here other contributors to this project are listed.