Skip to main content

The NetHack Learning Environment (NLE): a reinforcement learning environment based on NetHack

Project description

NetHack Learning Environment (NLE)


Twitter

The NetHack Learning Environment (NLE) is a Reinforcement Learning environment originally presented at NeurIPS 2020. This version of NLE is based on NetHack 3.6.7 and designed to provide a standard RL interface to the game, and comes with tasks that function as a first step to evaluate agents on this new environment.

NetHack is one of the oldest and arguably most impactful videogames in history, as well as being one of the hardest roguelikes currently being played by humans. It is procedurally generated, rich in entities and dynamics, and overall an extremely challenging environment for current state-of-the-art RL agents, while being much cheaper to run compared to other challenging testbeds. Through NLE, we wish to establish NetHack as one of the next challenges for research in decision making and machine learning.

You can read more about NLE in the NeurIPS 2020 paper, and about NetHack in its original README, at nethack.org, and on the NetHack wiki.

Example of an agent running on NLE

This version of NLE uses the Farama Organisation Gymnasium Environment APIs.

Getting started

Starting with NLE environments is extremely simple, provided one is familiar with other gym (or Gynmasium) / RL environments.

Installation

NLE requires python>=3.8, cmake>=3.18 to be installed and available both when building the package, and at runtime.

On MacOS, one can use Homebrew as follows:

$ brew install cmake

On a plain Ubuntu 18.04 distribution, cmake and other dependencies can be installed by doing:

# Python and most build deps
$ sudo apt-get install -y build-essential autoconf libtool pkg-config \
    python3-dev python3-pip python3-numpy git flex bison libbz2-dev

# recent cmake version
$ wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | sudo apt-key add -
$ sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'
$ sudo apt-get update && apt-get --allow-unauthenticated install -y \
    cmake \
    kitware-archive-keyring

Afterwards it's a matter of setting up your environment. We advise using a conda environment for this:

$ conda create -y -n nle python=3.8
$ conda activate nle
$ pip install nle

NOTE: If you want to extend / develop NLE, please install the package as follows:

$ git clone https://github.com/heiner/nle --recursive
$ pip install -e ".[dev]"
$ pre-commit install

Docker

We have provided some docker images. Please see the relevant README.

Trying it out

After installation, one can try out any of the provided tasks as follows:

>>> import gymnasium as gym
>>> import nle
>>> env = gym.make("NetHackScore-v0")
>>> env.reset()  # each reset generates a new dungeon
>>> env.step(1)  # move agent '@' north
>>> env.render()

NLE also comes with a few scripts that allow to get some environment rollouts, and play with the action space:

# Play NetHackStaircase-v0 as a human
$ python -m nle.scripts.play

# Use a random agent
$ python -m nle.scripts.play --mode random

# Play the full game using directly the NetHack internal interface
# (Useful for debugging outside of the gymnasium environment)
$ python -m nle.scripts.play --env NetHackScore-v0 # works with random agent too

# See all the options
$ python -m nle.scripts.play --help

Note that nle.scripts.play can also be run with nle-play, if the package has been properly installed.

Additionally, a TorchBeast agent is bundled in nle.agent together with a simple model to provide a starting point for experiments:

$ pip install "nle[agent]"
$ python -m nle.agent.agent --num_actors 80 --batch_size 32 --unroll_length 80 --learning_rate 0.0001 --entropy_cost 0.0001 --use_lstm --total_steps 1000000000

Plot the mean return over the last 100 episodes:

$ python -m nle.scripts.plot
                              averaged episode return

  140 +---------------------------------------------------------------------+
      |             +             +            ++-+ ++++++++++++++++++++++++|
      |             :             :          ++++++++||||||||||||||||||||||||
  120 |-+...........:.............:...+-+.++++|||||||||||||||||||||||||||||||
      |             :        +++++++++++++++||||||||||AAAAAAAAAAAAAAAAAAAAAA|
      |            +++++++++++++||||||||||||||AAAAAAAAAAAA|||||||||||||||||||
  100 |-+......+++++|+|||||||||||||||||||||||AA||||||||||||||||||||||||||||||
      |       +++|||||||||||||||AAAAAAAAAAAAAA|||||||||||+++++++++++++++++++|
      |    ++++|||||AAAAAAAAAAAAAA||||||||||||++++++++++++++-+:             |
   80 |-++++|||||AAAAAA|||||||||||||||||||||+++++-+...........:...........+-|
      | ++|||||AAA|||||||||||||||++++++++++++-+ :             :             |
   60 |++||AAAAA|||||+++++++++++++-+............:.............:...........+-|
      |++|AA||||++++++-|-+        :             :             :             |
      |+|AA|||+++-+ :             :             :             :             |
   40 |+|A+++++-+...:.............:.............:.............:...........+-|
      |+AA+-+       :             :             :             :             |
      |AA-+         :             :             :             :             |
   20 |AA-+.........:.............:.............:.............:...........+-|
      |++-+         :             :             :             :             |
      |+-+          :             :             :             :             |
    0 |-+...........:.............:.............:.............:...........+-|
      |+            :             :             :             :             |
      |+            +             +             +             +             |
  -20 +---------------------------------------------------------------------+
      0           2e+08         4e+08         6e+08         8e+08         1e+09
                                       steps

NLE Language Wrapper

We thank ngoodger for implementing the NLE Language Wrapper that translates the non-language observations from NetHack tasks into similar language representations. Actions can also be optionally provided in text form which are converted to the Discrete actions of the NLE.

NetHack Learning Dataset

The NetHack Learning Dataset (NLD) code now ships with NLE, allowing users to the load large-scale datasets featured in Dungeons and Data: A Large-Scale NetHack Dataset, while also generating and loading their own datasets.

import nle.dataset as nld

if not nld.db.exists():
    nld.db.create()
    # NB: Different methods are used for data based on NLE and data from NAO.
    nld.add_nledata_directory("/path/to/nld-aa", "nld-aa-v0")
    nld.add_altorg_directory("/path/to/nld-nao", "nld-nao-v0")

dataset = nld.TtyrecDataset("nld-aa-v0", batch_size=128, ...)
for i, mb in enumerate(dataset):
    foo(mb) # etc...

For information on how to download NLD-AA and NLD-NAO, see the dataset doc here.

Otherwise checkout the tutorial Colab notebook here.

Papers using the NetHack Learning Environment

Open a pull request to add papers.

Contributing

We welcome contributions to NLE. If you are interested in contributing please see this document.

Architecture

NLE is direct fork of NetHack and therefore contains code that operates on many different levels of abstraction. This ranges from low-level game logic, to the higher-level administration of repeated nethack games, and finally to binding of these games to Python gymnasium environment.

If you want to learn more about the architecture of nle and how it works under the hood, checkout the architecture document. This may be a useful starting point for anyone looking to contribute to the lower level elements of NLE.

Related Environments

Interview about the environment with Weights&Biases

Facebook AI Research’s Tim & Heiner on democratizing reinforcement learning research.

Interview with Weigths&Biases

Citation

If you use NLE in any of your work, please cite:

@inproceedings{kuettler2020nethack,
  author    = {Heinrich K{\"{u}}ttler and
               Nantas Nardelli and
               Alexander H. Miller and
               Roberta Raileanu and
               Marco Selvatici and
               Edward Grefenstette and
               Tim Rockt{\"{a}}schel},
  title     = {{The NetHack Learning Environment}},
  booktitle = {Proceedings of the Conference on Neural Information Processing Systems (NeurIPS)},
  year      = {2020},
}

If you use NLD or the datasets in any of your work, please cite:

@article{hambro2022dungeons,
  title={Dungeons and Data: A Large-Scale NetHack Dataset},
  author={Hambro, Eric and Raileanu, Roberta and Rothermel, Danielle and Mella, Vegard and Rockt{\"a}schel, Tim and K{\"u}ttler, Heinrich and Murray, Naila},
  journal={Advances in Neural Information Processing Systems},
  volume={35},
  pages={24864--24878},
  year={2022}
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nle-1.1.0.tar.gz (7.4 MB view details)

Uploaded Source

Built Distributions

nle-1.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

nle-1.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARM64

nle-1.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

nle-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.1 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64

nle-1.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

nle-1.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

nle-1.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

nle-1.1.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

nle-1.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

nle-1.1.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

File details

Details for the file nle-1.1.0.tar.gz.

File metadata

  • Download URL: nle-1.1.0.tar.gz
  • Upload date:
  • Size: 7.4 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for nle-1.1.0.tar.gz
Algorithm Hash digest
SHA256 3d6cd3d799181cf8bba6760b5d0cacb8886538b2a28943a79ca72afdcc4e0e3b
MD5 61ce3850a07debbcbb3a464e3cf3f979
BLAKE2b-256 4e00eb265c9cc99ca87c4427db102f8288f600b47c48a81b19dfd1febe8b15f1

See more details on using hashes here.

Provenance

The following attestation bundles were made for nle-1.1.0.tar.gz:

Publisher: test_and_deploy.yml on heiner/nle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nle-1.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for nle-1.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 16026327c513f8d76dc8362a9cc21743ad4654a799ecab38c044cb4a8fef936f
MD5 9ccf571cb13c967cc85d7a6552d978ad
BLAKE2b-256 fee10effec27135d0bd87613ff62be7a22fe2a5ca6107557f5b3e9cb7296672a

See more details on using hashes here.

Provenance

The following attestation bundles were made for nle-1.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: test_and_deploy.yml on heiner/nle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nle-1.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for nle-1.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 fd862b917dc0296f2c9a4a3888885c88bcb75b95413a6a2cd8fc0503e9b270f5
MD5 ad3e66e3250e0f2c63dfe6e61476f501
BLAKE2b-256 7e0f87c1be6e529c3b428534d694b34066c66febc55bf7e9aea3c13cbcf498b6

See more details on using hashes here.

Provenance

The following attestation bundles were made for nle-1.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: test_and_deploy.yml on heiner/nle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nle-1.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for nle-1.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b793546d797297e4e092de743e8c0d8caea017f9327ebd724a7f678c2b2604e8
MD5 b301abe2aa609d10150d5e49a63be9ab
BLAKE2b-256 64d5680f7fd0d013977ce83b3f5acd28f84ed0bad80c7195edcb907f52433729

See more details on using hashes here.

Provenance

The following attestation bundles were made for nle-1.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: test_and_deploy.yml on heiner/nle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nle-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for nle-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ffc6a02228979c84c8205e4a1921364d2324a1e6f7c6225ec4413b6d9bcfd7c1
MD5 5a143693ceaa564c625312b7bbb5b1ae
BLAKE2b-256 ce873c3b8762e4e3e2ae1d2fac17e7f980917fe5e35631ce6f3a7b579d7758f3

See more details on using hashes here.

Provenance

The following attestation bundles were made for nle-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: test_and_deploy.yml on heiner/nle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nle-1.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for nle-1.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 9505b12837aa74aefb6be458c04c18fe7692d361b0d67084a805b4fe6f4b3937
MD5 7b5f9a752ac7ef8ac504b2c443f54d67
BLAKE2b-256 06a97439075974a85d75792268eaf47d28a26bb4fdbbbf39ac261522e6c305c9

See more details on using hashes here.

Provenance

The following attestation bundles were made for nle-1.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: test_and_deploy.yml on heiner/nle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nle-1.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for nle-1.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 3c907fbe04a5ce75de49a89d934fec3a892ae73da474a9f77ab15e512c19be70
MD5 26d4499328d49b2059115ee8a56aa0e7
BLAKE2b-256 dd1759a5dc442ef38f451fd3b7f9283ab63524d3d72f6bfa5185899c275d10cc

See more details on using hashes here.

Provenance

The following attestation bundles were made for nle-1.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: test_and_deploy.yml on heiner/nle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nle-1.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for nle-1.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e356ac608cc63b2ffd666456d76f73fac582062e62c76d695aff1dadac0fedd8
MD5 7c70a6a75134d79e1e2bc534923e59f0
BLAKE2b-256 78bdd539fbd280e3c0da2e55fa536b80049a0c9de921ee89a6785b3044f9beed

See more details on using hashes here.

Provenance

The following attestation bundles were made for nle-1.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: test_and_deploy.yml on heiner/nle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nle-1.1.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for nle-1.1.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 6ebe07a8e5275befdbf5f3b1dee91a3e65c2f4e0ef0daf64082c005eb12eb86f
MD5 c09602b3940f7b2d35aced6b35d09ff8
BLAKE2b-256 17ee49114cdc070ea2e7d749b9fe038d838c889a9028cb553967c18525b3423f

See more details on using hashes here.

Provenance

The following attestation bundles were made for nle-1.1.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: test_and_deploy.yml on heiner/nle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nle-1.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for nle-1.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 754bf691f0fb00346bbb5875ab21e7ee829ea6a23cba631ab88e5c98d4c5d6be
MD5 3574ea2dd7dd46635449d2392b38c201
BLAKE2b-256 132fa68766f78c7a3533a48ab5137efdcfbef0f96e78d047ad1f954b3efaf1ee

See more details on using hashes here.

Provenance

The following attestation bundles were made for nle-1.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: test_and_deploy.yml on heiner/nle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nle-1.1.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for nle-1.1.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 7f8e8095bad4a86ab8a769ab51ba26e0165ff4816ab745091e296fec8f97e601
MD5 66d29b5591832cd5c9f27b0729fb8b1c
BLAKE2b-256 e4c598a15ad40e677d97b8c4e6f0482a6afb73fd5ba8931ae0c6e98a05463ca5

See more details on using hashes here.

Provenance

The following attestation bundles were made for nle-1.1.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: test_and_deploy.yml on heiner/nle

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page