Cogment Agent Toollbox
Project description
Agent Toolbox
Overview
The agent toolbox provides for the Environment a way to specify meta data in on the proto file. This helps to provide specification for the tensor. It is not mandatory to use meta_data to be compatible with the tools provided for the agents.
This tool generates tensors from the observation and action space.
Setup
- Add the Agent Toolbox in your Cogment project.
- Copy the google folder (TODO add the url there) in your Cogment project.
- In the orchestrator/Dockerfile add:
ADD google ./
On the Environment side
-
Include meta_data.proto in your own proto
-
Add the following metadata for the following :
- To remove some field from the tensor add the exclude_from_tensor metadata aside it :
[(meta_data.exclude_from_tensor) = true]
for instance if you have a field
float orientation = 1;
replace it by the following line:
float orientation = 1 [(meta_data.exclude_from_tensor) = true];
- The same way you can specify min or max, or both :
[(meta_data.min) = -1.0, (meta_data.max) = 1.0]
- It is also possible to specify an increment:
float angle = 1 [(meta_data.min) = -3.14159, (meta_data.max) = 3.14159, (meta_data.increment) = 0.3];
- It is also possible to provide a hint on how many the repeated field could be used with repeated_count_hint:
[(meta_data.repeated_count_hint) = 3]
On the Agent side
Import TensorGenerator class in your agent
- Create a generator
from agent_toolbox.tensor_generator import TensorGenerator
generator = TensorGenerator()
- Get the action size and observation size of the environment. These functions are uselfull to create the neural net with the right sizes.
generator = TensorGenerator()
state_size = generator.get_state_size()
action_size = generator.get_action_size()
- Compute state tensor from the environment observation or compute action tensor from an environment action
state = generator.compute_tensor(obs)
- Compute action from a tensor, useful function when the type of action is continuous
action_from_tensor = generator.compute_action_from_tensor(action_tensor)
Discrete Actions utils
- Consult all possible actions generated for a classification problem
generator.get_possible_actions()
- Returns the number of actions generated for a classification problem
classification_size = generator.get_classification_size()
equivalent to
len(generator.get_possible_actions())
- Compute a choice for a classification problem, from an action tensor. The returned value is an index in the possible action list.
generator.get_possible_actions()
choice = generator.compute_choice_from_action(action_tensor)
- Convert the selected action index in action required by the environment. The parameter
choice
is an index from 0 to n, n beinggenerator.get_classification_size()
.
generator.convert_choice_in_action(choice)
Examples
- Using action tensor directly, useful for continuous action type.
from agent_toolbox.tensor_generator import TensorGenerator
...
def decide(self, observation: cog_settings.actor_classes._otsa.observation_space):
# Create generator
generator = TensorGenerator()
# Transform the observation space in tensor
obs_tensor = generator.compute_tensor(observation)
print("Observation tensor: ", obs_tensor)
# Pick an action from a tensor
action_size = generator.get_action_size()
action_tensor = [random.random() for i in range(action_size)]
# Transform the tensor in action
action_from_tensor = generator.compute_action_from_tensor(action_tensor)
print("Action from tensor : ", action_from_tensor)
- Solving classification problem by using a list generated actions, useful for discrete action type.
def decide(self, observation: cog_settings.actor_classes._otsa.observation_space):
# Create generator
generator = TensorGenerator()
# Transform the observation space in tensor
obs_tensor = generator.compute_tensor(observation)
print("Observation tensor: ", obs_tensor)
# Pick an action from a classification problem
classification_size = generator.get_classification_size()
action_choice = random.randrange(classification_size)
# Transform the choice in action
action_from_choice = generator.convert_choice_in_action(
action_choice)
print("Action from choice : ", action_from_choice)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for cogment_agent_toolbox-0.0.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9b42d9533b39f2d2942695ea15125146a7602c69f9e1759e62c888ec1fc4fc75 |
|
MD5 | 43a3eb4dd141c4f4aabe68cfe723c2bf |
|
BLAKE2b-256 | 9068c48ee71ebb7d120d287d5b9dc556834a008eeb2133a1c3bd61daab7040a9 |