Lightweight environments to study hierarchical reasoning
Project description
HierarchyCraft - Environements builder for hierarchical ML research
HierarchyCraft
HierarchyCraft (hcraft for short) is a Python library designed to create arbitrary hierarchical environments that are compatible with both the OpenAI Gym Reinforcement Learning Framework and AIPlan4EU Unified Planning Framework. This library enables users to easily create complex hierarchical structures that can be used to test and develop various reinforcement learning or planning algorithms.
In environments built with HierarchyCraft the agent (player) has an inventory and can navigate into abstract zones that themselves have inventories.
Available actions of the agent are defined by transformations. Transformations are the core element of every HierarchyCraft environment, they define the actions that an agent can perform in the environment. At their core, transformations are simply changes to the state of the environment that can be triggered by the agent if the transformation is allowed in the current state.
Create your own tailored HierarchyCraft environments
You can use HierarchyCraft to create various custom hierarchical environments. As an example, here is a replica of the underlying hierarchy of the popular game Minecraft without the computationaly intensive 3D:
Play yourself!
pip install hcraft[gui]
hcraft minecraft
For more examples, see examples.
To create a HierarchyCraft environment, one need to understand transformations fully.
Each transformation is composed of three optional elements:
-
The first element is a destination zone, which specifies the zone to which the agent will move if the transformation changes the player position. This element is only necessary if the transformation changes the agent's position in the environment.
-
The second element is a list of zones to which the transformation is restricted. This element allows users to define which zones the agent can perform the transformation in, restricting the agent's actions to certain areas of the environment. If not specified, the transformation can be done anywhere.
-
The third element is a dictionary of inventory changes for the player, the current zone, the destination zone, or any specific zones. Inventory changes are defined as a dictionary mapping operations such as 'add', 'remove', 'min', 'max' on lists of stacks of items. This element allows users to define changes to any inventory that may result from the transformation, it also restricts the transformation to only be possible if the amount of the specified item is more that min and less than max. If unspecified, no changes are done, and 'min' will default to 0, 'max' to +inf.
For examples, see hcraft.transformation
Each HierarchyCraft environment is defined by a list of transformations and an initial state. The initial state defines the starting state of the environment, including the agent's position, inventory, and zones inventories. By combining transformations and an initial state, users can simply create complex hierarchical environments with a high degree of flexibility and control.
See hcraft.env
for a complete tutorial on creating custom environments.
See hcraft.state
for more details on the HierarchyCraft environements state.
Custom purposes for agents in HierarchyCraft environments
HierarchyCraft allows users to specify custom purposes for agents in their environments, composed of one or multiple tasks. This feature provides a high degree of flexibility and allows users to design environments that are tailored to specific applications or scenarios. This feature enables to study mutli-task or lifelong learning settings.
See hcraft.purpose
for more details.
Solving behavior for all tasks of most HierarchyCraft environments
HierarchyCraft also includes solving behaviors that can be used to generate actions from observations that will complete most tasks in any HierarchyCraft environment, including those designed by the user.
Solving behaviors are handcrafted, and may not work in some edge cases when transformations require items in specific zones. This feature makes it easy for users to obtain a strong baseline in their custom environments.
See hcraft.solving_behaviors
for more details.
Visualizing the underlying hierarchy of the environment (requirements graph)
HierarchyCraft gives the ability to visualize the hierarchy of the environment as a requirements graph. This graph provides a potentialy complex but complete representation of what is required to obtain each item or to go in each zone, allowing users to easily understand the structure of the environment and identify key items of the environment.
For example, here is the graph of the 'MiniCraftUnlock' environment where the goal is to open a door using a key:
And here is much more complex graph of the 'MineHcraft' environment shown previously:
See hcraft.requirements
for more details.
Installation
Using pip
Without optional dependencies:
pip install hcraft
With all optional dependencies:
pip install hcraft[all]
All hcraft environments can use a common graphical user interface. With gui requirements:
pip install hcraft[gui]
Environment can also be obtained throught the gym interface: With gym requirements:
pip install hcraft[gym]
Quickstart
See command line interface (CLI) options with:
hcraft --help
Using the programmatic interface:
from hcraft import MineHcraftEnv, get_human_action
env = MineHcraftEnv()
# or env: MineHcraftEnv = gym.make("MineHcraft-NoReward-v1")
n_episodes = 2
for _ in range(n_episodes):
env.reset()
done = False
total_reward = 0
while not done:
env.render()
action = get_human_action(env)
print(f"Human pressed: {env.world.transformations[action]}")
_observation, reward, done, _info = env.step(action)
total_reward += reward
print(f"SCORE: {total_reward}")
Run MineHcraft with autonomous agents
Using the programmatic interface, any HierarchyCraft environment can easily be interfaced with any agent.
import numpy as np
from hcraft import MineHcraftEnv
def random_legal_agent(observation, action_is_legal):
action = np.random.choice(np.nonzero(action_is_legal)[0])
return action
env = MineHcraftEnv(max_step=10)
done = False
observation = env.reset()
while not done:
action_is_legal = env.action_masks()
action = random_legal_agent(observation, action_is_legal)
_observation, _reward, done, _info = env.step(action)
Other hcraft environments
See hcraft.examples
.
from hcraft import TowerHcraft, RecursiveHcraft, RandomHcraft
tower_env = TowerHcraft(height=3, width=2)
# or tower_env = gym.make("TowerHcraft-v1", height=3, width=2)
recursive_env = RecursiveHcraft(n_items=6)
# or recursive_env = gym.make("RecursiveHcraft-v1", n_items=6)
random_env = RandomHcraft(n_items_per_n_inputs={0:2, 1:5, 2:10}, seed=42)
# or random_env = gym.make("RandomHcraft-v1", n_items_per_n_inputs={0:2, 1:5, 2:10}, seed=42)
Learn more in the documentation
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.