Gymie - Server

WebSocket server that exposes an API to train AI agents on OpenAI Gym and gym-api like environments such as Gym Retro or Unity ML-Agents, this last one with the help of gym wrapper

Content of this document

• Installation
• How to start the server
• API and how to consume it
  • List of methods exposed to the client
    • make
    • step
    • reset
    • close
    • observation_space
    • action_space
    • action_sample
  • Programmatic API
    • @override
    • start
• Testing Gymie
• Licence

Installation

Gymie can be installed using:

$ pip install gymie

or by cloning the repo and pip-installing in editable mode from the folder:

$ git clone https://github.com/jscriptcoder/Gymie-Server.git
Cloning into 'Gymie-Server'...
...

$ cd Gymie-Server/
$ pip install -e .
Obtaining file:///path/to/Gymie-Server
...
Successfully installed gymie

How to start the server

You can start the server from the command line:

$ python -m gymie --host 0.0.0.0 --port 5000
(84581) wsgi starting up on http://0.0.0.0:5000

or programmatically:

import gymie

gymie.start('localhost', 9000)

API and how to consume it

A client can communicate with Gymie via JSON, with the following format:

{
  "method": "api_method_name",
  "params": {
    "param1": "string",
    "param2": 6,
    "param3": true,
    "param4": []
  }
}

List of methods exposed to the client

• make: Instantiates an environment.

// Params:
{
  "env_id": "CartPole-v1",
  "seed":   0 // optional
}

// Response:
{
  "instance_id": "unique-id"
}

• step: Performs a step on the environment.

// Params:
{
  "instance_id": "instance-id"
  "action":      [1, 0, 1] // MultiBinary action
}

// Response:
[
  [...], // next state
  -2.0,  // reward
  false, // done
  {...}, // info
]

• reset: Resets the environment.

// Params:
{
  "instance_id": "instance-id"
}

// Response:
[...] // initial state

• close: Closes the environment.

// Params:
{
  "instance_id": "instance-id"
}

// Response:
true

• observation_space: Generates a dictionary with observation space info.

// Params:
{
  "instance_id": "instance-id"
}

// Response for Discreate observation space:
{
  "name": "Discreate",
  "n":    4
}

// Response for Box (Continuous) observation space:
{
  "name":  "Box",
  "shape": [3],
  "low":   [-5, -5, -5],
  "high":  [5, 5, 5]
}

// Response for MultiBinary observation space:
{
  "name":  "MultiBinary",
  "n":     5,
  "shape": [5]
}

// TODO MultiDiscrete

• action_space: Generates a dictionary with action space info.

// Params:
{
  "instance_id": "instance-id"
}

// Response for Discreate actions:
{
  "name": "Discreate",
  "n":    4
}

// Response for Box (Continuous) actions:
{
  "name":  "Box",
  "shape": [2],
  "low":   [-1, -1],
  "high":  [1, 1]
}

• action_sample: Generates a random action.

// Params:
{
 "instance_id": "instance-id"
}

// Response for Discrete actions:
2

// Response for Continuous actions:
[1.52, -3.67]

Programmatic API

• @override: Decorator to override internal functionality. It takes a string, function's name, as an argument. This is useful if we want to use different gym-like wrappers. For example, both Gym Retro and Unity ML-Agents have different ways to instantiate an environment. You can take a look at the tests to see how it's done for Gym Retro and Unity ML-Agents (with the help of gym-unity). At the moment there are two internal functions that can be overriden, get_env and process_step.

Signature:

def override(func_name: str) -> Callable

How to use:

import retro
from gymie import override
from gym_unity.envs import UnityToGymWrapper
from mlagents_envs.environment import UnityEnvironment, UnityEnvironmentException

@override('get_env')
def retro_get_env(env_id, seed=None):
    """Instantiates a Gym environment"""
    try:
        env = retro.make(game=env_id)
    except FileNotFoundError:
        raise EnvironmentNotFound
    else:
        if seed: 
            env.seed(seed)

        return env


@override('process_step')
def unity_process_step(step):
    """Does some processing of the step"""
    observation, reward, done, info = step
    return observation.tolist(), float(reward), done, {}

• start: This function takes two arguments, host and port, and starts the server, listening on ws://host:port

Signature:

def start (host: str = '0.0.0.0', port: int = 5000) -> None

How to use:

import gymie

gymie.start('localhost', 8080)

Testing Gymie

You can run all the tests by executing run_tests.sh script:

$ ./run_tests.sh

In order to run test_gymie_retro.py you need to have gym-retro package installed. For tests/test_gymie_unity.py, you need mlagents-envs and gym-unity.

License

MIT License - Copyright (c) 2020 Francisco Ramos