Agent Toolbox

Overview

The agent toolbox generates tensors from the observation and action space. These tensors or vectors can be used directly to Tensorflow or PyTorch.

Setup

pip install cogment_agent_toolbox

Use TensorGenerator class in your agent

• Create a generator

from agent_toolbox.tensor_generator import TensorGenerator
generator = TensorGenerator()

• Get the action size and observation size of the environment. These functions are uselfull to create the neural net with the right sizes.

generator = TensorGenerator()
state_size = generator.get_state_size()
action_size = generator.get_action_size()

• Compute state tensor from the environment observation or compute action tensor from an environment action

state = generator.compute_tensor(obs)

• Compute action from a tensor, useful function when the type of action is continuous

action_from_tensor = generator.compute_action_from_tensor(action_tensor)

Discrete Actions utils

• Consult all possible actions generated for a classification problem

generator.get_possible_actions()

• Returns the number of actions generated for a classification problem

classification_size = generator.get_classification_size()

equivalent to

len(generator.get_possible_actions())

• Compute a choice for a classification problem, from an action tensor. The returned value is an index in the possible action list. generator.get_possible_actions()

choice = generator.compute_choice_from_action(action_tensor)

• Convert the selected action index in action required by the environment. The parameter choice is an index from 0 to n, n being generator.get_classification_size().

generator.convert_choice_in_action(choice)

Examples

• Using action tensor directly, useful for continuous action type.

from agent_toolbox.tensor_generator import TensorGenerator

...

    def decide(self, observation: cog_settings.actor_classes._otsa.observation_space):

        # Create generator
        generator = TensorGenerator()

        # Transform the observation space in tensor
        obs_tensor = generator.compute_tensor(observation)
        print("Observation tensor: ", obs_tensor)

        # Pick an action from a tensor
        action_size = generator.get_action_size()
        action_tensor = [random.random() for i in range(action_size)]

        # Transform the tensor in action
        action_from_tensor = generator.compute_action_from_tensor(action_tensor)

        print("Action from tensor : ", action_from_tensor)

• Solving classification problem by using a list generated actions, useful for discrete action type.

    def decide(self, observation: cog_settings.actor_classes._otsa.observation_space):

        # Create generator
        generator = TensorGenerator()

        # Transform the observation space in tensor
        obs_tensor = generator.compute_tensor(observation)
        print("Observation tensor: ", obs_tensor)

        # Pick an action from a classification problem
        classification_size = generator.get_classification_size()
        action_choice = random.randrange(classification_size)

        # Transform the choice in action
        action_from_choice = generator.convert_choice_in_action(
            action_choice)

        print("Action from choice : ", action_from_choice)