A highly scalable and customizable safe reinforcement learning environment.
Project description
Safety-Gymnasium
Why Safety-Gymnasium?| Environments| Installation| Documentation| Design environments by yourself
This library is currently under heavy development - if you have suggestions on the API or use-cases you'd like to be covered, please open an github issue or reach out. We'd love to hear about how you're using the library.
Safety-Gymnasium is a highly scalable and customizable Safe Reinforcement Learning library, aiming to deliver a good view of benchmarking Safe Reinforcement Learning (Safe RL) algorithms and a more standardized setting of environments. We provide a set of standard API which is compatible with information on constraints. Users can explore new insights via an elegant code framework and well-designed environments.
Why Safety-Gymnasium?
Here we provide a table for comparison of Safety-Gymnasium and existing SafeRL Environments libraries.
SafeRL Envs |
Engine | Vectorized Environments |
New Gym API(3) | Vision Input |
---|---|---|---|---|
Safety Gym |
mujoco-py (1) |
❌ | ❌ | minimally supported |
safe-control-gym |
PyBullet | ❌ | ❌ | ❌ |
Velocity-Constraints(2) | N/A | ❌ | ❌ | ❌ |
mujoco-circle |
PyTorch | ❌ | ❌ | ❌ |
Safety Gymnaisum |
MuJoCo 2.3.0+ | ✅ | ✅ | ✅ |
(1): Maintenance (expect bug fixes and minor updates); the last commit is 19 Nov 2021. Safety Gym depends on mujoco-py
2.0.2.7, which was updated on Oct 12, 2019.
(2): There is no official library for speed-related environments, and its associated cost constraints are constructed from info. But the task is widely used in the study of SafeRL, and we encapsulate it in Safety-Gymnasium.
(3): In the gym 0.26.0 release update, a new API of interaction was redefined.
Environments
We designed a variety of safety-enhanced learning tasks and integrated the contributions of RL community:safety-velocity
, safety-run
, safety-circle
, safety-goal
, safety-button
, etc, leading to a unified safety-enhanced learning benchmark environment called Safety-Gymnasium.
Further, to facilitate the progress of community research, we redesigned Safety Gym and removed the dependency on mujoco-py
. We built it on top of MuJoCo and fixed some bugs, more specific bug report can refer to Safety Gym's BUG Report.
Here is a list of all the environments we support for now; some are being tested in our baselines, and we will gradually disclose it in the later updates.
Category | Task | Agent |
---|---|---|
Safe Navigation | Goal[012] | Point, Car, Racecar, Ant |
Button[012] | ||
Push[012] | ||
Circle[012] | ||
Safe Velocity | Velocity | HalfCheetah, Hopper, Swimmer, Walker2d, Ant, Humanoid |
Here is some pictures about tasks in Safe Navigation.
Agents
Coming soon… |
Coming soon… |
Tasks
Coming soon… |
Coming soon… |
Coming soon… |
Vision-base Safe RL
Vision-based safety reinforcement learning lacks realistic scenarios. Although the original Safety Gym
could minimally support visual input, the scenarios were too homogeneous. To facilitate the validation of visual-based safety reinforcement learning algorithms, we have developed a set of realistic vision-based SafeRL tasks, which are currently being validated on the baseline. In the later updates, we will release that part of the environment of Safety-Gymnasium.
For the appetizer, the images are as follows:
Environment Usage
Notes: We support explicitly express cost based on Gymnasium APIs.
import safety_gymnasium
env_name = 'SafetyPointGoal1-v0'
env = safety_gymnasium.make(env_name)
obs, info = env.reset()
terminated = False
while not terminated:
act = env.action_space.sample()
obs, reward, cost, terminated, truncated, info = env.step(act)
env.render()
Installation
Install from PyPi
pip install safety-gymnasium Notes:coming as soon as possible.
Install from source
conda create -n <virtual-env-name> python=3.8
conda activate <virtual-env-name>
git clone git@github.com:PKU-MARL/Safety-Gymnasium.git
cd Safety-Gymnasium
pip install -e .
Design environments by yourself
We construct a highly expandable framework of code so that you can easily comprehend it and design your own environments to facilitate your research with no more than 100 lines of code on average.
Here is a minimal example:
# import the objects you want to use
# or you can define specific objects by yourself, just make sure obeying our specification
from safety_gymnasium.assets.geoms import Apples
from safety_gymnasium.bases import BaseTask
# inherit the basetask
class MytaskLevel0(BaseTask):
def __init__(self, config):
super().__init__(config=config)
# define some properties
self.num_steps = 500
self.robot.placements = [(-0.8, -0.8, 0.8, 0.8)]
self.robot.keepout = 0
self.lidar_max_dist = 6
# add objects into environments
self.add_geoms(Apples(num=2, size=0.3))
self.specific_agent_config()
def calculate_reward(self):
# implement your reward function
# Note: cost calculation is based on objects, so it's automatic
reward = 1
return reward
def specific_agent_config(self):
# depending on your task
pass
def specific_reset(self):
# depending on your task
def specific_step(self):
# depending on your task
def build_goal(self):
# depending on your task
def update_world(self):
# depending on your task
@property
def goal_achieved(self):
# depending on your task
License
Safety-Gymnasium is released under Apache License 2.0.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.