Skip to main content

fastrl is a reinforcement learning library that extends Fastai. This project is not affiliated with fastai or Jeremy Howard.

Project description

fastrl

Note: This is a perminant location for fastrl version 2. Currently in a giant refactor. The previous source code can be found here.

CI Status pypi fastrl version Conda fastrl version Docker Image Latest Docker Image-Dev Latest

Anaconda-Server Badge fastrl python compatibility fastrl license

{% include warning.html content='Even before fastrl==2.0.0, all Models should converge reasonably fast, however HRL models DADS and DIAYN will need ' %}re-balancing and some extra features that the respective authors used.

Overview

Here is change

Fastai for computer vision and tabular learning has been amazing. One would wish that this would be the same for RL. The purpose of this repo is to have a framework that is as easy as possible to start, but also designed for testing new agents.

Documentation is being served at https://josiahls.github.io/fastrl/ from documentation directly generated via nbdev in this repo.

Current Issues of Interest

Data Issues

  • data and async_data are still buggy. We need to verify that the order that the data being returned is the best it can be for our models. We need to make sure that "dones" are returned and that there are new duplicate (unless intended)
  • Better data debugging. Do environments skips steps correctly? Do n_steps work correct?

Whats new?

As we have learned how to support as many RL agents as possible, we found that fastrl==1.* was vastly limited in the models that it can support. fastrl==2.* will leverage the nbdev library for better documentation and more relevant testing. We also will be building on the work of the ptan1 library as a close reference for pytorch based reinforcement learning APIs.

1 "Shmuma/Ptan". Github, 2020, https://github.com/Shmuma/ptan. Accessed 13 June 2020.

Install

PyPI (Not implemented yet)

Placeholder here, there is no pypi package yet. It is recommended to do traditional forking.

(For future, currently there is no pypi persion)pip install fastrl==2.0.0 --pre

Conda (Not implimented yet)

conda install -c fastchan -c josiahls fastrl

source activate fastrl && python setup.py develop

Docker (highly recommend)

Install: Nvidia-Docker

Install: docker-compose

docker-compose pull && docker-compose up

Contributing

After you clone this repository, please run nbdev_install_git_hooks in your terminal. This sets up git hooks, which clean up the notebooks to remove the extraneous stuff stored in the notebooks (e.g. which cells you ran) which causes unnecessary merge conflicts.

Before submitting a PR, check that the local library and notebooks match. The script nbdev_diff_nbs can let you know if there is a difference between the local library and the notebooks.

  • If you made a change to the notebooks in one of the exported cells, you can export it to the library with nbdev_build_lib or make fastai2.
  • If you made a change to the library, you can export it back to the notebooks with nbdev_update_lib.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

fastrl-0.0.34.tar.gz (32.1 kB view details)

Uploaded Source

Built Distribution

fastrl-0.0.34-py3-none-any.whl (38.6 kB view details)

Uploaded Python 3

File details

Details for the file fastrl-0.0.34.tar.gz.

File metadata

  • Download URL: fastrl-0.0.34.tar.gz
  • Upload date:
  • Size: 32.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.2 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.0 CPython/3.9.6

File hashes

Hashes for fastrl-0.0.34.tar.gz
Algorithm Hash digest
SHA256 f08b543f469f0c95fbc70ed033aa84fc10b955f3aebf7d3be5794b70d7bba7e6
MD5 958330c8a467cf59178aa6c1e7cb042a
BLAKE2b-256 8197138b262dfdfb40b48a2e767447f63300174f3205921288242b8c5f5c5b77

See more details on using hashes here.

File details

Details for the file fastrl-0.0.34-py3-none-any.whl.

File metadata

  • Download URL: fastrl-0.0.34-py3-none-any.whl
  • Upload date:
  • Size: 38.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.2 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.0 CPython/3.9.6

File hashes

Hashes for fastrl-0.0.34-py3-none-any.whl
Algorithm Hash digest
SHA256 814b6fcd8bec5250b60e34fa026d89c407fac8c1034b88f1a7cb1026b3895cbe
MD5 cf893ad14b2751ee41059c58d95b7f4f
BLAKE2b-256 072e0e8a487ca4ce5c24bd907635fbe8aaa1113d1284760e5c74f9d71e7b7875

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page