mercury-rl

Introduction

Welcome to mercury-rl, a library for offline deep reinforcement learning. This library offers classic implementations of state-of-the-art algorithms such as Conservative Q-Learning (CQL) and its variations, including Deep Q-Network (DQN), Actor-Critic (AC), Trust Region Policy Optimization (TRPO), and Proximal Policy Optimization (PPO).

Our goal is to provide a toolkit to develop, experiment, and deploy reinforcement learning models efficiently. mercury-rl aims to make your offline RL journey smoother and more productive.

What is Offline Reinforcement Learning?

Offline reinforcement learning (offline RL) is a subfield of reinforcement learning where the agent is trained using a static dataset of interactions with the environment. On the other hand, in online reinforcement learning the agent continuously interacts with the environment to collect data and update its policy.

Offline RL is particularly useful in scenarios where real-time interaction with the environment is costly, risky, or impractical. Some examples include healthcare, robotics, and autonomous driving, where it is often not feasible to let an untrained agent explore freely.

Key Differences Between Offline RL and Online RL

Figure taken from this post

Components

Algorithms

The mercury-rl library implements a range of state-of-the-art algorithms for both discrete and continuous control tasks. Below is a summary of the key algorithms included in the library:

Requirements

mercury-rl development requires the following software installed:

• Python 3.6 or higher

Once the developer has checked out the source code from the repository, any changes to the code can be done through the creation of a new branch.

Install

To install mercury-rl you only need a pip-install:

Datio

pip install --user mercury-rl

Local

You'll need to configure your Artifactory credentials. If you don't know how, you can find a mini tutorial on our Mercury’s developer handbook.

pip install mercury-rl --extra-index-url https://\${ARTIFACTORY_BOT_BASIC_AUTH}@artifactory.globaldevtools.bbva.com/artifactory/api/pypi/gl-datio-runtime-pypi-local/simple

Exploratory notebooks

from mercury.rl import create_tutorials

create_tutorials('mercury_tutorials')

The code above creates a local folder named mercury_tutorials and places a collection of notebooks inside showing different mercury.rl features.

Contributing

Want to contribute to mercury-rl? More info about it on Mercury’s developer handbook.

Powered by Mercury.