minihack 0.1.0b0

('MiniHack The Planet: ', 'A Sandbox for Open-Ended Reinforcement Learning Research')

MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research

MiniHack is a sandbox framework for easily designing rich and diverse environments for Reinforcement Learning (RL) research. Based on the game of NetHack, arguably the hardest grid-based game in the world, MiniHack uses the NetHack Learning Environment (NLE) to communicate with the game and provide a convenient interface for customly created RL tesbeds.

MiniHack already comes with a large list of challenging tasks. However, it is primarily built for easily designing new ones. The motivation behind MiniHack is to be able to perform RL experiments in a controlled setting while being able to increasingly scale the complexity of the tasks.

To this end, MiniHack leverages the description files of NetHack. The description files (or des-files) are human-readable specifications of levels: distributions of grid layouts together with monsters, objects on the floor, environment features (e.g. walls, water, lava), etc. The developers of NetHack created a special domain-specific language for describing the levels of the game, called des-file format. The des-files can be compiled into binary using the NetHack level compiler, and MiniHack maps them to Gym environments. For more information on des-files, we refer users to our brief overview, detailed visually-aided tutorial, or interactive notebook.

MiniHack documentation will walk you through everything you need to know, step-by-step, including information on how to get started with MiniHack, configure environments or design new ones, train baseline agents, use our APi, and much more.

Installation

MiniHack is available on pypi and can be installed as follows:

pip install minihack

We advise using a conda environment for this:

conda create -n minihack python=3.8
conda activate minihack
pip install minihack

NOTE: NLE requires cmake>=3.15 to be installed when building the package. Checkout out here how to install it in MacOS and Ubuntu 18.04. Windows users should use Docker.

NOTE: Baseline agents have separate installation instructions. See here for more details.

Extending MiniHack

If you wish to extend MiniHack, please install the package as follows:

git clone https://github.com/facebookresearch/minihack
cd minihack
pip install -e ".[dev]"
pre-commit install

Docker

We have provided several Dockerfiles for building images with pre-installed MiniHack. Please follow the instructions described here.

Trying out MiniHack

MiniHack uses the popular Gym interface for the interactions between the agent and the environment. A pre-registered MiniHack environment can be used as follows:

import gym
import minihack
env = gym.make("MiniHack-River-v0")
env.reset() # each reset generates a new environment instance
env.step(1)  # move agent '@' north
env.render()

To see the list of all MiniHack environments, run:

python -m minihack.scripts.env_list

The following scripts allow to play MiniHack environments with a keyboard:

# Play the MiniHack in the Terminal as a human
python -m minihack.scripts.play --env MiniHack-River-v0

# Use a random agent
python -m minihack.scripts.play --env MiniHack-River-v0  --mode random

# Play the MiniHack with graphical user interface (gui)
python -m minihack.scripts.play_gui --env MiniHack-River-v0

NOTE: If the package has been properly installed one could run the scripts above with mh-envs, mh-play, and mh-guiplay commands.

Baseline Agents

Several baseline agents are included as part of MiniHack, which can be installed and used as follows:

• a TorchBeast agent is bundled in minihack.agent.polybeast together with a simple model to provide a starting point for experiments. To install and train this agent, first install torchbeast be following the instructions here, then use the following commands:

$ pip install ".[polybeast]"
$ python3 -m minihack.agent.polybeast.polyhydra env=small_room_random learning_rate=0.0001 use_lstm=true total_steps=1000000

• An RLlib agent is provided in minihack.agent.rllib, with a similar model to the torchbeast agent. This can be used to try out a variety of different RL algorithms - several examples are provided. To install and train this agent use the following commands:

$ pip install ".[rllib]"
$ python -m minihack.agent.rllib.train algo=dqn

More information on running these agents, and instructions on how to reproduce the results of the MiniHack paper, can be found in this document. We also provide baseline learning curves of the MiniHack paper in Weights&Biases format for all of our experiments. They can be accessed here.

Contributions and Maintenance

We welcome contributions to MiniHack. If you are interested in contributing please see this document. MiniHack's maintenance plan can be found here.

Citation

If you use MiniHack in your work, please cite:

@misc{samvelyan2021minihack,
  title={MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research},
  author={Mikayel Samvelyan and Robert Kirk and Vitaly Kurin and Jack Parker-Holder and Minqi Jiang and Eric Hambro and Fabio Petroni and Heinrich Kuttler and Edward Grefenstette and Tim Rockt{\"a}schel},
  year={2021},
  url={https://openreview.net/forum?id=skFwlyefkWJ}
}

If you use MiniHack's interface on environments ported from other benchmarks, please cite the original paper as well: MiniGrid (see license, bibtex), Boxoban (see license, bibtex).