Skip to main content

Cleanest Deep Reinforcement Learning Implementation Based on Web MVC

Project description

CircleCI Maintainability Test Coverage


Clean deep reinforcement learning codes based on Web MVC architecture with complete unit tests


Implementing deep reinforcement learning algorithms is easy to make up messy codes because interaction loop between an environment and an agent requires a lot of dependencies among classes. Even deep learning requires special skills to build clean codes.

To think out of the box, Web engineers spent years on studying MVC (model-view-controller) architecture to build systems with tidy codes to handle interaction between Web and users. Here, I found that this MVC architecture is very useful insight even for deep reinforcement learning implementation. MVC provides a direction to an architecture with less dependencies, which would be nicer for unit testing.



You can use docker to setup and run experiments.

$ ./scripts/

Once you built the container, you can start a container with nvidia runtime via ./scripts/

$ ./scripts/
root@a84ab59aa668:/home/app#  ls
Dockerfile    example.confing.json  graphs            mvc      scripts  tests
LICENSE     examples     logs                  requirements.txt  tools


You need to install packages written in requirements.txt and tensorflow.

$ pip install -r requirements.txt
$ pip install tensorflow-gpu tensorflow-probability-gpu

If you have a problem of installing tensorflow probability, check tensorflow version.


For academic usage, we provide baseline implementations that you might need to compare.

  • Proximal Policy Optimization
  • Deep Deterministic Policy Gradients
  • Soft Actor-Critic

Ant performance

Each point represents an average evaluation reward of 10 episodes. Pretty much same performance has been achieved as a paper of Soft Actor-Critic.


$ python -m examples.ppo --env Ant-v2



$ python -m examples.ddpg --env Ant-v2



$ python -m examples.sac --env Ant-v2 --reward-scale 5




unit testing

To gurantee code quality, all functions and classes including neural networks must have unit tests.

Following command runs all unit tests under tests directory.

$ ./

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

mvc-1.0.0-py3-none-any.whl (38.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page