bluesky-gym 0.2.0

A gymnasium style environment for standardized Reinforcement Learning research in Air Traffic Management.

BlueSky-Gym

A gymnasium style library for standardized Reinforcement Learning research in Air Traffic Management developed in Python. Build on BlueSky and The Farama Foundation's Gymnasium

An example trained agent attempting the merge environment available in BlueSky-Gym.

For a complete list of the currently available environments click here

Installation

Update 27 February 2025: There is currently a bug when pip installing BlueSky-Simulator, which causes the pip install to fail on most machines (see issue). For now, users can clone the repository linked in this branch and pip install the requirements.txt file to circumvent this problem. This branch contains a local, barebones, version of BlueSky-Simulator from which the required functionality is retrieved.

pip install bluesky-gym

Note that the pip package is bluesky-gym, for usage however, import as bluesky_gym.

Usage

Using the environments follows the standard API from Gymnasium, an example of which is given below:

import gymnasium as gym
import bluesky_gym
bluesky_gym.register_envs()

env = gym.make('MergeEnv-v0', render_mode='human')

obs, info = env.reset()
done = truncated = False
while not (done or truncated):
    action = ... # Your agent code here
    obs, reward, done, truncated, info = env.step(action)

Additionally you can directly use algorithms from standardized libraries such as Stable-Baselines3 or RLlib to train a model:

import gymnasium as gym
import bluesky_gym
from stable_baselines3 import DDPG
bluesky_gym.register_envs()

env = gym.make('MergeEnv-v0', render_mode=None)
model = DDPG("MultiInputPolicy",env)
model.learn(total_timesteps=2e6)
model.save()

Contributing and Assistance

If you would like to contribute to BlueSky-Gym or need assistance in setting up or creating your own environments, do not hesitate to open an issue or reach out to one of us via the BlueSky-Gym Discord. Additionally you can have a look at the roadmap for inspiration on where you can contribute and to get an idea of the direction BlueSky-Gym is going.

Citing

If you use BlueSky-Gym in your work, please cite it using:

@misc{bluesky-gym,
  author = {Groot, DJ and Leto, G and Vlaskin, A and Moec, A and Ellerbroek, J},
  title = {BlueSky-Gym: Reinforcement Learning Environments for Air Traffic Applications},
  year = {2024},
  journal = {SESAR Innovation Days 2024},
}

List of publications & preprints using BlueSky-Gym (please open a pull request to add missing entries):

• missing entry