Language Wrapper for the NetHack Learning Environment (NLE)
Project description
Nethack Learning Environment Language Wrapper
Language Wrapper for the Nethack Learning Environment (NLE) and MiniHack
Description
The wrapper inherits from the Gym Wrapper and translates the non-language observations from NLE into similar language representations. Actions can also be optionally provided in text form which are converted to the Discrete actions of the NLE.
Inventory:
a: a blessed +1 mace (weapon in hand)
b: a +0 robe (being worn)
c: a blessed +0 small shield (being worn)
d: 4 potions of holy water
e: a clove of garlic
f: a sprig of wolfsbane
g: a spellbook of stone to flesh
h: a spellbook of identify
Stats:
Strength:15/15
Dexterity:10
Constitution:12
Intelligence:12
Wisdom:18
Charisma:9
Depth:1
Gold:0
HP:14/14
Energy:6/6
AC:7
XP:1/0
Time:1
Position:46|14
Hunger:Not Hungry
Monster Level:0
Encumbrance:Unemcumbered
Dungeon Number:0
Level Number:1
Score:0
Alignment:Neutral
Condition:None
Cursor:Yourself a priestess
Observation:
vertical closed door far westnorthwest
horizontal wall near north and northwest
vertical wall very near northeast and east
vertical closed door very near eastnortheast
southeast corner very near southeast
horizontal wall very near south and southwest
tame kitten adjacent northeast
Message:
Hello Agent, welcome to NetHack! You are a neutral human Priestess.
Observations
The environment converts the NLE observations: glyphs
, blstats
, tty_chars
, inv_letters
, inv_strs
and tty_cursor
to language equivalents.
text_glyphs
: A compressed textual representation of the surroundings.
dark area far west
vertical wall near east and southeast
horizontal wall near south and southwest
horizontal closed door near southsouthwest
black onyx ring near westsouthwest
doorway near west
egg very near east
horizontal wall adjacent north, northeast, and northwest
tame little dog adjacent southwest
Corresponding to the following visual display
---------
.....@.%|
|...d...|
|.......|
|=......|
----+----
text_message
: Current message. Same asmessage
from NLE however also includes menus when present.
Aloha Agent, welcome to NetHack! You are a neutral female human Tourist.
text_blstats
: Text version of the bottom-line stats and auxiliary stats include with NLE.
Strength:11/11
Dexterity:12
Constitution:14
Intelligence:16
Wisdom:9
Charisma:14
Depth:1
Gold:241
HP:10/10
Energy:2/2
AC:10
XP:1/0
Time:1
Position:48|2
Hunger:Not Hungry
Monster Level:0
Encumbrance:Unemcumbered
Dungeon Number:0
Level Number:1
Score:0
Alignment:Neutral
Condition:None
text_inventory
: Current inventory with letters.
$: 241 gold pieces
a: 22 +2 darts (at the ready)
b: 6 uncursed food rations
c: 3 uncursed tripe rations
d: an uncursed egg
e: 2 uncursed fortune cookies
f: 2 uncursed potions of extra healing
g: 2 uncursed scrolls of magic mapping
h: 2 blessed scrolls of magic mapping
i: an uncursed +0 Hawaiian shirt (being worn)
j: an expensive camera (0:68)
k: an uncursed credit card
text_cursor
: Description of glyph currently under cursor.
Yourself a tourist
Actions
Actions are by default text actions like wait
, apply
, north
ect. The corresponding key-presses are supported as well, e.g. west
is the same as h
and kick
is the same as ^d
. Alternatively the standard discrete action space from NLE can be used by passing use_language_action=False
to the wrapper.
Getting Started
Requirements
Requires python>=3.7
and cmake>=3.15
.
To install
CMake can be installed on macos using homebrew
brew install cmake
Alternatively, and for other platforms follow the instructions at https://cmake.org/install/
Installation
To use the environment you can install from the PyPI.
pip install nle-language-wrapper
Development
For development on the environment clone the repository and install in development mode.
git clone https://github.com/ngoodger/nle-language-wrapper --recursive
pip install -e ".[dev]"
To update the library with changes to the C++ recompile by running
python -m setup develop
Usage
The wrapper can be used simply by instantiating the base environment from NLE or MiniHack and passing it to the wrapper constructor. E.g. From python
import nle
from nle_language_wrapper import NLELanguageWrapper
env = NLELanguageWrapper(gym.make("NetHackChallenge-v0"))
obsv = env.reset()
obsv, reward, done, info = env.step("wait")
Alternatively to utilize the discrete actions rather than language actions specify use-text-action
.
env = NLELanguageWrapper(gym.make("NetHackChallenge-v0"), use_language_action=text)
obsv = env.reset()
wait_action = 17
obsv, reward, done, info = env.step(wait_action)
Manual play
To directly interact with an NLE or MiniHack environment run the following script.
python -m nle_language_wrapper.scripts.play
Agent
There is a Sample Factory based agent included achieving 730 reward after 1B frames. This agent uses a small transformer model to encode the language observations for the policy model and value function model. The algorithm used is Asynchronous Proximal Policy Optimization (APPO) described in Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning.
Hardware Requirements
The default configuration was tested on an Nvidia 3090 with 24Gbyte and a Ryzen 1700 CPU. Training runs at approximately 5k/FPS. To train on less GPU memory a smaller model could be configured, or a smaller max token length or batch size could be used.
--transformer_hidden_size" (default=64)
--transformer_hidden_layers" (default=2)
--transformer_attention_heads (default=2)
--max_token_length (default=256)
--batch_size (default=1024)
Running the agent
There is a pre-trained agent included with the project.
python nle_language_wrapper/agents/sample_factory/enjoy.py \
--env nle_language_env \
--encoder_custom nle_language_transformer_encoder \
--experiment nle_language_agent \
--algo APPO \
--fps 1
Training the agent
To train a new agent simply run the following module and the set the experiment name to the desired value.
python nle_language_wrapper/agents/sample_factory/train.py \
--env nle_language_env \
--encoder_custom nle_language_transformer_encoder \
--experiment nle_language_agent_1 \
--algo APPO \
--batch_size 2048 \
--num_envs_per_worker 24 \
--num_workers 8 \
--reward_scale 0.1
Licence
MIT License
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for nle-language-wrapper-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | ae3668568f28eb411a438fae7322933acb8b766dceacfcf23163ab8453f5b916 |
|
MD5 | 7e50fa7f2962b05fcd885c1f0296d923 |
|
BLAKE2b-256 | cf480b342102bd1bf1de88d57bd6dcd36d39e23fac1874660651090eb8b90734 |