Gym JSBSim environment

# JSBSim Aircraft Simulator for Open AI Gym Environment

## Are we able to learn an aircraft to fly (or taxi) ?

This project aims at creating realistic open AI GYM environements using the open source Flight Dynamic Model JSBSim. Our work is initially based on the Gor-Ren repository (https://github.com/Gor-Ren/gym-jsbsim) on which we made several modifications:

• Remove links to Flight-Gear to focus only on JSBSim and GYM environment making.
• A process to add extra features (like delta_heading for exemple) and the capability to call a dedicated function to update this attribute every time steps.
• A feature to allow a set_state() and a get_state() to go back to a previous JSBSim state (very usefull if you want to use MCTS, ICT, IW and planning /scheduling approaches)
• An autopilot for ground procedure (A320) based on 2 PIDs to manage throttles and brakes, to make the aircraft taxiing at a specific velocity.
• A complete taxi environement with access to runaway path attributes (like distances and angles to next points) to train an aircraft to taxi autonomously.
• Allow to define discrete GYM attributes.
• ...

## Installation

### Mac OS / Linux

The package is directly installable with pip typing the following command :

pip install gym-jsbsim


### Windows

First, you have to install the Shapely library (issue #39) with :

• ##### Python 3.6 :
pip install https://download.lfd.uci.edu/pythonlibs/t7epjj8p/Shapely-1.6.4.post2-cp36-cp36m-win_amd64.whl

• ##### Python 3.7 :
pip install https://download.lfd.uci.edu/pythonlibs/t7epjj8p/Shapely-1.6.4.post2-cp37-cp37m-win_amd64.whl


Then, you can install the package with the pip install command :

pip install gym-jsbsim


### Sources

If you want to build from the source, clone the repository using the following command as we have submodules:

git clone --recurse-submodules https://github.com/galleon/gym-jsbsim.git


## Environments

So far, is available:

• Taxi with AutoPilot task: GymJsbsim-TaxiapControlTask-v0: The aircraft should follow a predifined trajectory on the runaway. The aircraft velocity is manage by an autopilot.
• Approach Task: GymJsbsim-ApproachControlTask-v0: The aircraft should decrese its altitude (it is still a draft environement and not realitic in regards to appraoch procedure).

import gym
import gym_jsbsim

env.reset()
done = False

while not done:
action = env.action_space.sample()
state, reward, done, _ = env.step(action)


In this task, the aircraft should perform a stable steady flight following its initial heading and altitude. Every 150 seconds, a new target heading is set. At each time the target heading is more and more complicated to reach, starting with a delta of ±10° (random sign), following with ±20°, then ±30° and so on.

A terminal state is reached:

• If the target heading is not reached with an accuracy of 10° during the 150 seconds.
• If the aircraft is more than 100 feet away from its target altitude when the heading target changes.

The input state set is a vector of 9 parameters:

state_var = [c.delta_altitude, # the delta altitude between the aircraft and target altitude
c.velocities_v_down_fps, # the vertical velocity of the aircraft
c.velocities_vc_fps, # the air velocity of the aircraft


The action set is a vector of 4 parameters:

action_var = [c.fcs_aileron_cmd_norm,
c.fcs_elevator_cmd_norm,
c.fcs_rudder_cmd_norm,
c.fcs_throttle_cmd_norm]


The reward is computed as a function of:

• altitude deviation from target
• roll angle
• speed
• acceleration perceived by the pilot in the cockpit

import gym
import gym_jsbsim

env.reset()
done = False

while not done:
action = env.action_space.sample()
state, reward, done, _ = env.step(action)


This is the same as the Heading Task, but the target altitude also changes every 150 seconds. The altitude changes are: first ±100 feet (random sign), then ±200 feet, ±300 feet, and so on, but modulo 5000 feet and with a minimum target altitude of 3000 feet.

import gym
import gym_jsbsim

env.reset()
done = False

while not done:
action = env.action_space.sample()
state, reward, done, _ = env.step(action)


In this environnement, the aircraft should behave on ground and following a specific path, trying to be closest as possible to the runaway centerline.

For this environement, we have extract a path from the AMDB files of blagnac airport:

and extracted from this path a list of geodesic coordinates that the aircraft should follow.

As state set for this environment, we compute every timestep, the distance (di) and angle (ai) to the next 4 path points according to location of the aircraft (ie: d1 to d4 and a1 to a4)

The full state set is:

state_var = [c.velocities_vc_fps,
c.shortest_dist,
c.d1, c.d2, c.d3, c.d4,
c.a1, c.a2, c.a3, c.a4]


The action set is define with 3 parameters that correspond to throttle, brake and steer commands:

action_var = [c.fcs_steer_cmd_norm,
c.fcs_center_brake_cmd_norm,
c.fcs_throttle_cmd_norm]


the reward is simply computed with the distance to the centerline

The scenario is over if the aircraft overtake the centerline from 10 meters or more.

import gym
import gym_jsbsim

env.reset()
done = False

while not done:
action = env.action_space.sample()
state, reward, done, _ = env.step(action)


This is the same Taxi environement with an autopilot to manage the aircraft velocity. The velocity during straight line is set to 20 knots and the velocity during turn is set to 7 knots.

The state set is unchange:

state_var = [c.velocities_vc_fps,
c.shortest_dist,
c.d1, c.d2, c.d3, c.d4,
c.a1, c.a2, c.a3, c.a4]


but the action set is therefor focus on the steering command only:

action_var = [c.fcs_steer_cmd_norm]


the reward is unchanged and computed with the distance to the centerline

## Test

You could run a random agent with

python test_agent.py


## Results

Open AI GYM: https://github.com/openai/gym

Gor-Ren Gym-Jsbsim repo: https://github.com/Gor-Ren/gym-jsbsim

## Project details

Uploaded source
Uploaded py2 py3