Skip to main content

Gym environments which provide offline RL datasets collected on the TriFinger system.

Project description

TriFinger RL Datasets

This repository provides offline reinforcement learning datasets collected on the real TriFinger platform and in a simulated version of the environment. The paper "Benchmarking Offline Reinforcement Learning on Real-Robot Hardware" provides more details on the datasets and benchmarks offline RL algorithms on them. All datasets are available with camera images as well.

More detailed information about the simulated environment, the datasets and on how to run experiments on a cluster of real TriFinger robots can be found in the documentation.

Some of the datasets were used during the Real Robot Challenge 2022.

Installation

To install the package run with python 3.8 in the root directory of the repository (we recommend doing this in a virtual environment):

pip install --upgrade pip  # make sure the most recent version of pip is installed
pip install .

Usage

This section provides short examples of how to load datasets and evaluate a policy in simulation. More details on how to work with the datasets can be found in the documentation.

Loading a dataset

The datasets are accessible via gym environments which are automatically registered when importing the package. They are automatically downloaded when requested and stored in ~/.trifinger_rl_datasets as Zarr files by default (see the documentation for custom paths to the datasets). The code for loading the datasets follows the interface suggested by D4RL and extends it where needed.

As an alternative to the automatic download, the datasets can also be downloaded manually from the Edmond repository.

The datasets are named following the pattern trifinger-cube-task-source-type-v0 where task is either push or lift, source is either sim or real and type can be either mixed, weak-n-expert or expert.

By default the observations are loaded as flat arrays. For the simulated datasets the environment can be stepped and visualized. Example usage (also see demo/load_dataset.py):

import gymnasium as gym

import trifinger_rl_datasets

env = gym.make(
    "trifinger-cube-push-sim-expert-v0",
    visualization=True,  # enable visualization
)

dataset = env.get_dataset()

print("First observation: ", dataset["observations"][0])
print("First action: ", dataset["actions"][0])
print("First reward: ", dataset["rewards"][0])

obs, info = env.reset()
truncated = False

while not truncated:
    obs, rew, terminated, truncated, info = env.step(env.action_space.sample())

Alternatively, the observations can be obtained as nested dictionaries. This simplifies working with the data. As some parts of the observations might be more useful than others, it is also possible to filter the observations when requesting dictionaries (see demo/load_filtered_dicts.py):

    # Nested dictionary defines which observations to keep.
    # Everything that is not included or has value False
    # will be dropped.
    obs_to_keep = {
        "robot_observation": {
            "position": True,
            "velocity": True,
            "fingertip_force": False,
        },
        "object_observation": {"keypoints": True},
    }
    env = gym.make(
        args.env_name,
        # filter observations,
        obs_to_keep=obs_to_keep,
    )

All datasets come in two versions: with and without camera observations. The versions with camera observations contain -image in their name. Despite PNG image compression they are more than one order of magnitude bigger than the imageless versions. To avoid running out of memory, a part of a dataset can be loaded by specifying a range of timesteps:

env = gym.make(
    "trifinger-cube-push-real-expert-image-v0",
    disable_env_checker=True
)

# load only a subset of obervations, actions and rewards
dataset = env.get_dataset(rng=(1000, 2000))

The camera observations corresponding to this range are then returned in dataset["images"] with the following dimensions:

n_timesteps, n_cameras, n_channels, height, width = dataset["images"].shape

Evaluating a policy in simulation

This package contains an executable module trifinger_rl_datasets.evaluate_sim, which can be used to evaluate a policy in simulation. As arguments it expects the task ("push" or "lift") and a Python class that implements the policy, following the PolicyBase interface:

python3 -m trifinger_rl_datasets.evaluate_sim push my_package.MyPolicy

For more options see --help.

How to cite

The paper "Benchmarking Offline Reinforcement Learning on Real-Robot Hardware" introducing the datasets was published at ICLR 2023:

@inproceedings{
guertler2023benchmarking,
title={Benchmarking Offline Reinforcement Learning on Real-Robot Hardware},
author={Nico G{\"u}rtler and Sebastian Blaes and Pavel Kolev and Felix Widmaier and Manuel Wuthrich and Stefan Bauer and Bernhard Sch{\"o}lkopf and Georg Martius},
booktitle={The Eleventh International Conference on Learning Representations },
year={2023},
url={https://openreview.net/forum?id=3k5CUGDLNdd}
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

trifinger_rl_datasets-1.0.3.tar.gz (1.1 MB view details)

Uploaded Source

Built Distribution

trifinger_rl_datasets-1.0.3-py3-none-any.whl (1.1 MB view details)

Uploaded Python 3

File details

Details for the file trifinger_rl_datasets-1.0.3.tar.gz.

File metadata

  • Download URL: trifinger_rl_datasets-1.0.3.tar.gz
  • Upload date:
  • Size: 1.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.8.10

File hashes

Hashes for trifinger_rl_datasets-1.0.3.tar.gz
Algorithm Hash digest
SHA256 2687c99e53922d42dcf0af88e310f59039680a416e9859cb0e36aec8be7b01e0
MD5 5ce1487cf1a55ce9800b411834fa902f
BLAKE2b-256 ac37b0d29413dd67bf55b1e426122565ff0f0b8b9d22102dcfe1f9e8d64febe4

See more details on using hashes here.

File details

Details for the file trifinger_rl_datasets-1.0.3-py3-none-any.whl.

File metadata

File hashes

Hashes for trifinger_rl_datasets-1.0.3-py3-none-any.whl
Algorithm Hash digest
SHA256 ea8647dfdffe03ad9984fec4b59b9692ce037b2d0d9708992e0f28f5975036ae
MD5 75ec334b4c3aee835d4e2b01045924fc
BLAKE2b-256 36c843f6eea2bfb25677b3847814a46b7c93bdf81bf717a9d5ded7f9be5f5810

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page