gymnasium-search-race 1.0.0

A reinforcement learning environment for the Search Race CG puzzle based on Gymnasium

Gymnasium Search Race

Gymnasium environment for the Search Race CodinGame optimization puzzle.

https://github.com/user-attachments/assets/1862b04b-9e33-4f55-a309-ad665a1db2f1

Action Space	Box([-1, 0], [1, 1], float64)
Observation Space	Box([0, 0, 0, 0, 0, 0, 0, -1, -1, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1], float64)
import	gymnasium.make("gymnasium_search_race:gymnasium_search_race/SearchRace-v1")

Installation

To install gymnasium-search-race with pip, execute:

pip install gymnasium_search_race

From source:

git clone https://github.com/Quentin18/gymnasium-search-race
cd gymnasium-search-race/
pip install -e .

Environment

Action Space

The action is a ndarray with 2 continuous variables:

• The rotation angle between -18 and 18 degrees, normalized between -1 and 1.
• The thrust between 0 and 200, normalized between 0 and 1.

Observation Space

The observation is a ndarray of 10 continuous variables:

• 1 if the next checkpoint is the last one, 0 otherwise.
• The x and y coordinates of the next checkpoint.
• The x and y coordinates of the checkpoint after next checkpoint.
• The x and y coordinates of the car.
• The horizontal speed vx and vertical speed vy of the car.
• The facing angle of the car.

The values are normalized between 0 and 1, or -1 and 1 if negative values are allowed.

Reward

The goal is to visit all checkpoints as quickly as possible, as such the agent is penalised with a reward of -0.1 for each timestep. When a checkpoint is visited, the agent is awarded with a reward of 1000/total_checkpoints.

Starting State

The starting state is generated by choosing a random CodinGame test case.

Episode End

The episode ends if either of the following happens:

1. Termination: The car visit all checkpoints before the time is out.
2. Truncation: Episode length is greater than 600.

Arguments

• test_id: test case id to generate the checkpoints (see choices here). The default value is None which selects a test case randomly when the reset method is called.

import gymnasium as gym

gym.make("gymnasium_search_race:gymnasium_search_race/SearchRace-v1", test_id=1)

Version History

• v1: Add boolean to indicate if the next checkpoint is the last checkpoint in observation
• v0: Initial version

Discrete environment

The SearchRaceDiscrete environment is similar to the SearchRace environment except the action space is discrete.

import gymnasium as gym

gym.make("gymnasium_search_race:gymnasium_search_race/SearchRaceDiscrete-v0", test_id=1)

Action Space

There are 6 discrete actions corresponding to the combinations of angles -18, 0 and 18 degrees and thrust 0 and 200.

Usage

You can use RL Baselines3 Zoo to train and evaluate agents:

pip install rl_zoo3

Train an Agent

The hyperparameters are defined in hyperparams/ppo.yml.

To train a PPO agent for the Search Race game, execute:

python -m rl_zoo3.train \
  --algo ppo \
  --env gymnasium_search_race/SearchRace-v1 \
  --tensorboard-log logs \
  --eval-freq 10000 \
  --eval-episodes 10 \
  --gym-packages gymnasium_search_race \
  --conf-file hyperparams/ppo.yml \
  --progress

Enjoy a Trained Agent

To see a trained agent in action on random test cases, execute:

python -m rl_zoo3.enjoy \
  --algo ppo \
  --env gymnasium_search_race/SearchRace-v1 \
  --n-timesteps 10000 \
  --deterministic \
  --gym-packages gymnasium_search_race \
  --load-best \
  --progress

Run Test Cases

To run test cases with a trained agent, execute:

python -m scripts.run_test_cases \
  --path rl-trained-agents/ppo/best_model.zip \
  --record-video \
  --record-metrics

Tests

To run tests, execute:

pytest

Citing

To cite the repository in publications:

@misc{gymnasium-search-race,
  author = {Quentin Deschamps},
  title = {Gymnasium Search Race},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Quentin18/gymnasium-search-race}},
}

References

• Gymnasium
• RL Baselines3 Zoo
• Stable Baselines3
• CGSearchRace

Author

Quentin Deschamps