cogment-agent-toolbox 0.0.4

Cogment Agent Toollbox

Agent Toolbox

Overview

The agent toolbox generates tensors from the observation and action space. These tensors or vectors can be used directly to Tensorflow or PyTorch.

Setup

pip install cogment_agent_toolbox

Use TensorGenerator class in your agent

• Create a generator

from agent_toolbox.tensor_generator import TensorGenerator
generator = TensorGenerator()

• Get the action size and observation size of the environment. These functions are uselfull to create the neural net with the right sizes.

generator = TensorGenerator()
state_size = generator.get_state_size()
action_size = generator.get_action_size()

• Compute state tensor from the environment observation or compute action tensor from an environment action

state = generator.compute_tensor(obs)

• Compute action from a tensor, useful function when the type of action is continuous

action_from_tensor = generator.compute_action_from_tensor(action_tensor)

Discrete Actions utils

• Consult all possible actions generated for a classification problem

generator.get_possible_actions()

• Returns the number of actions generated for a classification problem

classification_size = generator.get_classification_size()

equivalent to

len(generator.get_possible_actions())

• Compute a choice for a classification problem, from an action tensor. The returned value is an index in the possible action list. generator.get_possible_actions()

choice = generator.compute_choice_from_action(action_tensor)

• Convert the selected action index in action required by the environment. The parameter choice is an index from 0 to n, n being generator.get_classification_size().

generator.convert_choice_in_action(choice)

Examples

• Using action tensor directly, useful for continuous action type.

from agent_toolbox.tensor_generator import TensorGenerator

...

    def decide(self, observation: cog_settings.actor_classes._otsa.observation_space):

        # Create generator
        generator = TensorGenerator()

        # Transform the observation space in tensor
        obs_tensor = generator.compute_tensor(observation)
        print("Observation tensor: ", obs_tensor)

        # Pick an action from a tensor
        action_size = generator.get_action_size()
        action_tensor = [random.random() for i in range(action_size)]

        # Transform the tensor in action
        action_from_tensor = generator.compute_action_from_tensor(action_tensor)

        print("Action from tensor : ", action_from_tensor)

• Solving classification problem by using a list generated actions, useful for discrete action type.

    def decide(self, observation: cog_settings.actor_classes._otsa.observation_space):

        # Create generator
        generator = TensorGenerator()

        # Transform the observation space in tensor
        obs_tensor = generator.compute_tensor(observation)
        print("Observation tensor: ", obs_tensor)

        # Pick an action from a classification problem
        classification_size = generator.get_classification_size()
        action_choice = random.randrange(classification_size)

        # Transform the choice in action
        action_from_choice = generator.convert_choice_in_action(
            action_choice)

        print("Action from choice : ", action_from_choice)