Skip to main content

Gym-Ignition: A toolkit for developing OpenAI Gym environments simulated with Ignition Gazebo.

Project description

gym-ignition

WhatWhyHowDemoSetupCitation


CI/CD Docker Images CI PyPI CD Codacy Badge
gym-ignition
gym-ignition-nightly


What

Gym-Ignition provides the capability of creating reproducible robotics environments for reinforcement learning research.

The project is composed of the following components:

Component Description
ignition A set of classes to interface gympp with the new Ignition Gazebo simulator, part of the Ignition Robotics suite.
plugins Ignition Gazebo plugins.
gym_ignition Python package for creating OpenAI Gym environments. Environments can be implemented either in C++ using gympp or in Python using the SWIG binded classes of the ignition component.
gym_ignition_data SDF and URDF models and Gazebo worlds.
gympp An experimental C++ port of the OpenAI Gym interfaces, used to create pure C++ environments.

Why

Refer to the Citation for the extended rationale behind this project.

TL;DR

We designed Gym-Ignition driven by the following reasons:

  • Advances in RL research are pushed by the development of more complex environments, and vice versa.
  • There's no standard framework in the robotics community for creating simulated robotic environments.
  • Environments that can be transferred from simulation to reality with minimal changes do not exist.
  • Alternative solutions are not developed by roboticists for roboticist, and therefore they do not use familiar tools.
  • Existing robotics environments are typically difficult to adapt to run on different physics engines and different robotic platforms.
  • Only few solutions offer realistic rendering capabilities.

How

This project interfaces with the new generation of the Gazebo simulator, called Ignition Gazebo. It is part of the new Ignition Robotics suite developed by Open Robotics.

Ignition Robotics is currently under heavy development and is not yet stable. Though, it already offers enough functionalities for this project's aims:

  • Simulator-as-a-library
  • New modular architecture
  • C++ utilities developed with a robotic mindset
  • New abstractions of physics engines and rendering engines that exploit runtime plugins
  • Full support of DART and coming support of bullet3
  • Support of distributed simulations
  • Well maintained and packaged
  • Ignition Fuel database to download models and worlds

Features

At the time of writing, Gym-Ignition offers the following features:

  • Environments compatible with OpenAI Gym
  • Worlds and models are SDF descriptions
  • Reproducibility guarantees
  • Accelerated and multiprocess execution
  • Environments are a combination of three elements:
    • Task: the logic of the decision-making problem. It defines how to process the given action, how to calculate the reward, how to structure the observation, and how to reset the environment. Task objects are robot-independent and runtime-independent.
    • Robot: unified interface to access to robot resources. It is used to gather data and send commands to either simulated or real robot in a seamless way.
    • Runtime: implements the actual step of the environment. Simulated runtimes step the simulator, real-time runtimes deal with real-time execution constraints. A Task object can be executed by any runtime without any change.
  • Experimental support to create C++ environments

Demo

We provide two different methods to test Gym-Ignition without the need to install it locally:

  1. Colab notebook: and run the example
  2. Docker image:
    docker pull diegoferigo/gym-ignition:latest
    pip3 install rocker
    
    # Intel GPU
    rocker --x11 diegoferigo/gym-ignition ./github/examples/python/launch_cartpole.py
    
    # Nvidia GPU
    rocker --x11 --nvidia diegoferigo/gym-ignition ./github/examples/python/launch_cartpole.py
    

Setup

The setup instructions expect a Ubuntu distribution with at least Python 3.6. Gym-Ignition is compatible also with other distributions (and, also, other OSs) under the assumption that the Ignition Robotics suite can be installed either from repos or source. Though, to keep the instructions simple, we only report the steps for the Ubuntu distro.

The process is different whether you're an user that wants to create environments using Gym-Ignition or you are a developer that wants to edit the Python and C++ upstream code.

Execute all the setup commands in the same terminal.

Common Steps

  1. Install the supported version of Ignition Gazebo:

    sudo sh -c 'echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-stable `lsb_release -cs` main" > /etc/apt/sources.list.d/gazebo-stable.list'
    wget http://packages.osrfoundation.org/gazebo.key -O - | sudo apt-key add -
    sudo apt update
    sudo apt install ignition-gazebo2
    

    We currently require the development version, that will be ABI-compatible with the upcoming Citadel release. Refer to the official documentation for more detailed information.

  2. Create a Python virtual environment as follows:

    sudo apt install virtualenv
    virtualenv -p python3.6 $HOME/venv
    source $HOME/venv/bin/activate
    

User setup

Check that the last release includes a wheel. Then install Gym-Ignition with pip3 install gym-ignition. After this step, you should be able to execute the example launch_cartpole.py.

Developer setup

  1. Install gcc 8 with apt install gcc-8. Export the following environment variables to enable it temporarily:

    export CC=gcc-8
    export CXX=g++-8
    
  2. Install SWIG with apt install swig

  3. Clone this repository

  4. Build and install the CMake project

    mkdir build
    cd build
    cmake -DCMAKE_INSTALL_PREFIX=<installprefix> ..
    cmake --build .
    cmake --build . --target install
    
  5. Install the Python package in editable mode:

    pip3 install -e .
    
  6. Export the following environment variable:

    # C++ bindings
    export PYTHONPATH=<installprefix>/lib/python3.6/site-packages
    

After these steps, you should be able to execute the example launch_cartpole.py.

Unstable builds

Gym-Ignition still doesn't have a steady release cycle strategy. This project targets mainly research, and its development is very active. In order to quickly deliver new features, we do our best to have a fast release cycle.

If you find interesting PRs that are not yet included in the most recent release, you can get the latest unstable version as follows:

  1. User installation: install gym-ignition-nightly. Be sure that the last release includes a wheel.

  2. Developer installation: check-out the devel branch after cloning the repository.

Citation

@inproceedings{ferigoGymIgnition2019,
  title = {Gym-Ignition: Reproducible Robotic Simulations for Reinforcement Learning},
  author = {Ferigo, Diego and Traversaro, Silvio and Pucci, Daniele},
  booktitle = {RSS 2019 Workshop on Closing the Reality Gap in Sim2real Transfer for Robotic Manipulation},
  year = {2019},
}

paper.pdf


Disclaimer: Gym-Ignition is an independent project and is not related by any means to OpenAI and Open Robotics.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for gym-ignition-nightly, version 1.0b1.dev64
Filename, size File type Python version Upload date Hashes
Filename, size gym_ignition_nightly-1.0b1.dev64-cp36-cp36m-manylinux1_x86_64.whl (1.0 MB) File type Wheel Python version cp36 Upload date Hashes View hashes
Filename, size gym-ignition-nightly-1.0b1.dev64.tar.gz (136.8 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page