Skip to main content

High fidelity simulated environments for reinforcement learning

Project description

Holodeck

Holodeck Video

Read the docs badge Build Status

Holodeck is a high-fidelity simulator for reinforcement learning built on top of Unreal Engine 4.

Features

  • 7+ rich worlds for training agents in, and many scenarios for those worlds
  • Linux and Windows support
  • Easily extend and modify training scenarios
  • Train and control more than one agent at once
  • Simple, OpenAI Gym-like Python interface
  • High performance - simulation speeds of up to 2x real time are possible. Performance penalty only for what you need
  • Run headless or watch your agents learn

Installation

pip install holodeck

(requires Python 3)

See Installation for complete instructions (including Docker).

Documentation

Usage Overview

Holodeck's interface is similar to OpenAI's Gym.

We try and provide a "batteries included" approach to minimize the configuration you have to do.

Here is a quick example using the DefaultWorlds package:

import holodeck

# Load the environment. This environment contains a UAV in a city.
env = holodeck.make("UrbanCity-MaxDistance")

# You must call `.reset()` on a newly created environment before ticking/stepping it
env.reset()                         

# The UAV takes 3 torques and a thrust as a command.
command = [0, 0, 0, 100]   

for i in range(30):
    state, reward, terminal, info = env.step(command)  
  • state: dict of sensor name to the sensor's value (nparray).
  • reward: the reward received from the previous action
  • terminal: indicates whether the current state is a terminal state.
  • info: contains additional environment specific information.

If you want to access the data of a specific sensor, import sensors and retrieving the correct value from the state dictionary:

from holodeck.sensors import Sensors

print(state[Sensors.LOCATION_SENSOR])

Multi Agent-Environments

Holodeck supports multi-agent environments.

Calls to step only provide an action for the main agent, and then tick the simulation.

act provides a persistent action for a specific agent, and does not tick the simulation. After an action has been provided, tick will advance the simulation forward. The action is persisted until another call to act provides a different action.

env = holodeck.make('CyberPunkCity-Follow')
env.reset()

# Provide an action for each agent
env.act('uav0', np.array([0, 0, 0, 100]))
env.act('nav0', np.array([0, 0, 0]))

# Advance the simulation
for i in range(300):
    # The action provided above is repeated
    s = env.tick()

You can access the reward, terminal and location for a multi agent environment as follows:

s['uav0'][Sensors.REWARD]
s['uav0'][Sensors.TERMINAL]
s['uav0'][Sensors.LOCATION_SENSOR]

(uav0 comes from the scenario configuration file)

Running Holodeck Headless

Holodeck can run headless with GPU accelerated rendering. See Using Holodeck Headless

Citation:

@misc{HolodeckPCCL,
  Author = {Joshua Greaves and Max Robinson and Nick Walton and Mitchell Mortensen and Robert Pottorff and Connor Christopherson and Derek Hancock and Jayden Milne and David Wingate},
  Title = {Holodeck: A High Fidelity Simulator},
  Year = {2018},
}

Holodeck is a project of BYU's Perception, Cognition and Control Lab (https://pcc.cs.byu.edu/).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for holodeck, version 0.3.0
Filename, size File type Python version Upload date Hashes
Filename, size holodeck-0.3.0-py3-none-any.whl (35.9 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size holodeck-0.3.0.tar.gz (32.1 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page