JoyRL

JoyRL is a parallel reinforcement learning library based on PyTorch and Ray. Unlike existing RL libraries, JoyRL is helping users to release the burden of implementing algorithms with tough details, unfriendly APIs, and etc. JoyRL is designed for users to train and test RL algorithms with only hyperparameters configuration, which is mush easier for beginners to learn and use. Also, JoyRL supports plenties of state-of-art RL algorithms including RLHF(core of ChatGPT)(See algorithms below). JoyRL provides a modularized framework for users as well to customize their own algorithms and environments.

Install

⚠️ Note that donot install JoyRL through any mirror image!!!

# you need to install Anaconda first
conda create -n joyrl python=3.10
conda activate joyrl
pip install -U joyrl

Torch install:

# CPU
pip install torch==2.2.1 torchvision==0.17.1 torchaudio==2.2.1
# CUDA 11.8
pip install torch==2.2.1 torchvision==0.17.1 torchaudio==2.2.1 --index-url https://download.pytorch.org/whl/cu118
# CUDA 12.1
pip install torch==2.2.1 torchvision==0.17.1 torchaudio==2.2.1 --index-url https://download.pytorch.org/whl/cu121

Usage

Quick Start

the following presents a demo to use joyrl. As you can see, first create a yaml file to config hyperparameters, then run the command as below in your terminal. That's all you need to do to train a DQN agent on CartPole-v1 environment.

joyrl --yaml ./presets/ClassControl/CartPole-v1/CartPole-v1_DQN.yaml

or you can run the following code in your python file.

import joyrl
if __name__ == "__main__":
    print(joyrl.__version__)
    yaml_path = "./presets/ClassControl/CartPole-v1/CartPole-v1_DQN.yaml"
    joyrl.run(yaml_path = yaml_path)

Documentation

More tutorials and API documentation are hosted on JoyRL docs or JoyRL 中文文档.

Algorithms

Name	Reference	Author
Q-learning	RL introduction	johnjim0816
Sarsa	RL introduction	johnjim0816
DQN	DQN Paper	johnjim0816
Double DQN	DoubleDQN Paper	johnjim0816
Dueling DQN	DuelingDQN Paper	johnjim0816
NoisyDQN	NoisyDQN Paper	johnjim0816
DDPG	DDPG Paper	johnjim0816
TD3	TD3 Paper	johnjim0816
A2C/A3C	A3C Paper	johnjim0816
PPO	PPO Paper	johnjim0816
SoftQ	SoftQ Paper	johnjim0816

Why JoyRL?

RL Platform	# of Alg. ⁽¹⁾	Custom Env	Async Training	RNN Support	Multi-Head Observation	Backend
Baselines	9	:heavy_check_mark: (gym)	:x:	:heavy_check_mark:	:x:	TF1
Stable-Baselines	11	:heavy_check_mark: (gym)	:x:	:heavy_check_mark:	:x:	TF1
Stable-Baselines3	7	:heavy_check_mark: (gym)	:x:	:x:	:heavy_check_mark:	PyTorch
Ray/RLlib	16	:heavy_check_mark:	:heavy_check_mark:	:heavy_check_mark:	:heavy_check_mark:	TF/PyTorch
SpinningUp	6	:heavy_check_mark: (gym)	:x:	:x:	:x:	PyTorch
Dopamine	7	:x:	:x:	:x:	:x:	TF/JAX
ACME	14	:heavy_check_mark: (dm_env)	:x:	:heavy_check_mark:	:heavy_check_mark:	TF/JAX
keras-rl	7	:heavy_check_mark: (gym)	:x:	:x:	:x:	Keras
cleanrl	9	:heavy_check_mark: (gym)	:x:	:x:	:x:	poetry
rlpyt	11	:x:	:x:	:heavy_check_mark:	:heavy_check_mark:	PyTorch
ChainerRL	18	:heavy_check_mark: (gym)	:x:	:heavy_check_mark:	:x:	Chainer
Tianshou	20	:heavy_check_mark: (Gymnasium)	:x:	:heavy_check_mark:	:heavy_check_mark:	PyTorch
JoyRL	11	:heavy_check_mark: (Gymnasium)	:heavy_check_mark:	:heavy_check_mark:	:heavy_check_mark:	PyTorch

Here are some other highlghts of JoyRL:

Provide a series of Chinese courses JoyRL Book (with the English version in progress), suitable for beginners to start with a combination of theory

Contributors

John Jim

Peking University

Qi Wang

Shanghai Jiao Tong University

Yiyuan Yang

University of Oxford

Metadata

Release files for joyrl 0.6.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for joyrl 0.6.8
File	Size	Uploaded
joyrl-0.6.8.tar.gz	79.2 kB	Dec 19, 2024	Details

Built distribution (wheel)

Table of built distributions (wheels) for joyrl 0.6.8
File	Interpreter	ABI	Platform	Reset
joyrl-0.6.8-py3-none-any.whl 108.5 kB Dec 19, 2024	Python 3	none	any	Details

Total release size: 187.6 kB

Release files / joyrl-0.6.8.tar.gz

Download URL	joyrl-0.6.8.tar.gz
Size	79.2 kB
Tags	Source
SHA-256 checksum How to use checksums	`20d338cc0fddd4fe4f2db7cc62c676be5cd58f67ef81ea4ed7dcc2a81b8dcace`
BLAKE2b-256 checksum How to use checksums	`ba6c6db9da741fd78b5fee299199caeb12cbc2229391d02b23cacfdf32d56947`
Upload date	Dec 19, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/5.1.0 CPython/3.10.14`

Release files / joyrl-0.6.8-py3-none-any.whl

Download URL	joyrl-0.6.8-py3-none-any.whl
Size	108.5 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`234f02e300e16b73eb42c384fbcbad8024c5d3268dd85ae1a76e166d33988cf7`
BLAKE2b-256 checksum How to use checksums	`02f21b6bb45e32e6ecfc73bdb8b78c3066989e78810050bfae4e503c1f3de8e3`
Upload date	Dec 19, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/5.1.0 CPython/3.10.14`