mlgame3d 0.1.1

A framework for playing Unity games with Python agents using ML-Agents Environment.

MLGame3D

A framework for playing Unity games with Python MLPlay classes using ML-Agents for communication with Unity.

Installation

pip install mlgame3d

Usage

Command-line Interface

MLGame3D provides a command-line interface that allows you to launch games directly from the command line:

python -m mlgame3d [options] <Unity game executable>

Options include:

• --version, -v: Display version information
• --help, -h: Display help information (built-in argparse option)
• --no-graphics, -ng: Run the Unity simulator in no-graphics mode
• --worker-id, -w: Set the worker ID for running multiple environments simultaneously (default: 0)
• --base-port, -p: Set the base port (default: None, will use Unity default port)
• --seed, -s: Set the random seed (default: 0)
• --timeout, -t: Set the timeout for waiting for environment connection (default: 60 seconds)
• --episodes, -e: Set the number of episodes to run (default: 5)
• --fps, -f: Set the rendering frame rate (default: 30)
• --ai, -i: Control mode for an instance (auto-numbered). Can be specified multiple times. Each occurrence is equivalent to --ai1, --ai2, etc. in order.
• --ai1, -i1: Control mode for instance 1. Can be a path to a Python file containing an MLPlay class, 'hidden', or 'manual' (default).
• --ai2, -i2: Control mode for instance 2. Can be a path to a Python file containing an MLPlay class, 'hidden', or 'manual' (default).
• --ai3, -i3: Control mode for instance 3. Can be a path to a Python file containing an MLPlay class, 'hidden', or 'manual' (default).
• --ai4, -i4: Control mode for instance 4. Can be a path to a Python file containing an MLPlay class, 'hidden', or 'manual' (default).
• --game-param, -gp: Game parameter in the format KEY VALUE. Can be specified multiple times for different parameters.

Examples:

# Connect to an already running Unity editor
python -m mlgame3d

# Launch a Unity game and run 10 episodes
python -m mlgame3d --episodes 10 path/to/your/game.exe
# Or using short options
python -m mlgame3d -e 10 path/to/your/game.exe

# Run without graphics
python -m mlgame3d --no-graphics path/to/your/game.exe
# Or using short options
python -m mlgame3d -ng path/to/your/game.exe

# Set random seed and worker ID
python -m mlgame3d -s 42 -w 1 path/to/your/game.exe

# Use 2 AI instances, with a custom MLPlay for the first one and manual control for the second one
python -m mlgame3d -i examples/simple_mlplay.py -i manual path/to/your/game.exe

# Use 3 AI instances: first controlled by MLPlay, second hidden, third controlled by another MLPlay
python -m mlgame3d -i mlplay1.py -i hidden -i mlplay2.py path/to/your/game.exe

# Specify control modes for specific player positions
python -m mlgame3d -i1 mlplay1.py -i2 manual -i3 mlplay2.py -i4 hidden path/to/your/game.exe

# Pass game parameters (checkpoint_count and checkpoint_mode)
python -m mlgame3d -i examples/simple_mlplay.py -gp checkpoint_count 10 -gp checkpoint_mode random path/to/your/game.exe

Code Interface

You can also use this framework in your Python code:

from mlgame3d.game_env import GameEnvironment
from mlgame3d.mlplay import RandomMLPlay
from mlgame3d.game_runner import GameRunner

# Create the environment with controlled players
env = GameEnvironment(
    file_name="YourUnityGame.exe",  # Or None to connect to a running Unity editor
    worker_id=0,
    no_graphics=False,
    controlled_players=[0, 1],  # Control P1 and P2
    control_modes=["mlplay", "mlplay"],  # Both controlled by MLPlay
    game_parameters=[("checkpoint_count", 10), ("checkpoint_mode", "random")]  # Game parameters
)

# Get information about the action space for each behavior
action_space_info_p1 = env.get_action_space_info(env.behavior_names[0])
action_space_info_p2 = env.get_action_space_info(env.behavior_names[1])

# Create MLPlay instances
mlplay1 = RandomMLPlay(action_space_info_p1, name="MLPlay1")
mlplay2 = RandomMLPlay(action_space_info_p2, name="MLPlay2")

# Create a mapping from MLPlay index to behavior name
mlplay_to_behavior_map = {
    0: env.behavior_names[0],  # First MLPlay controls first behavior
    1: env.behavior_names[1]   # Second MLPlay controls second behavior
}

# Create a game runner
runner = GameRunner(
    env=env,
    mlplays=[mlplay1, mlplay2],
    max_episodes=5,
    render=True,
    mlplay_timeout=0.1,  # Timeout for MLPlay actions in seconds
    game_parameters={"checkpoint_count": 10, "checkpoint_mode": "random"},
    mlplay_to_behavior_map=mlplay_to_behavior_map
)

# Run the game
runner.run()

# Close the environment
env.close()

Creating Custom MLPlay Classes

You can create a standalone MLPlay class without inheriting from any base class. This approach is simple and flexible.

Requirements for MLPlay class:

1. The class must be named MLPlay
2. The class must implement __init__, update, and reset methods

Here's an example of a minimal MLPlay class:

import numpy as np
from typing import Dict, Any

class MLPlay:
    def __init__(self, action_space_info=None):
        # Initialize your MLPlay instance
        pass
        
    def reset(self):
        # Reset your MLPlay instance for a new episode
        pass
        
    def update(self, observations, reward=0.0, done=False, info=None):
        # Process observations and choose an action
        # This is a simple example that returns a random 2D movement vector
        action = np.random.uniform(-1, 1, 2)
        
        # Normalize the action vector
        if np.linalg.norm(action) > 0:
            action = action / np.linalg.norm(action)
            
        return action

Loading External MLPlay Files

You can create custom MLPlay classes in separate Python files and load them at runtime using the command-line interface. The framework will automatically find and instantiate the MLPlay class in the file.

Requirements for external MLPlay files:

1. The file must contain a class named MLPlay
2. The class must implement __init__, update, and reset methods

Example of an external MLPlay file (simple_mlplay.py):

import numpy as np
from typing import Dict, Any

class MLPlay:
    def __init__(self, action_space_info=None):
        self.step_counter = 0
        
    def reset(self):
        self.step_counter = 0
        
    def update(self, observations, reward=0.0, done=False, info=None):
        self.step_counter += 1
        
        # Alternate between different actions
        if self.step_counter % 2 == 0:
            return np.array([1.0, 0.0])  # Move right
        else:
            return np.array([0.0, 1.0])  # Move forward

You can then use these MLPlay classes with the command-line interface:

python -m mlgame3d -i simple_mlplay.py path/to/your/game.exe

Or specify a specific player position:

python -m mlgame3d -i1 simple_mlplay.py path/to/your/game.exe

Or load them programmatically:

from mlgame3d.mlplay_loader import create_mlplay_from_file

# Create an MLPlay instance from an external file
mlplay = create_mlplay_from_file("simple_mlplay.py", action_space_info)

Framework Structure

• game_env.py: Provides the GameEnvironment class for communicating with Unity games
• mlplay.py: Provides the RandomMLPlay class for generating random actions
• game_runner.py: Provides the GameRunner class for running games and collecting statistics
• mlplay_loader.py: Provides functionality for loading MLPlay classes from external Python files
• __main__.py: Provides the command-line interface
• examples/: Contains example MLPlay implementations

Notes

• Make sure your Unity game has integrated the ML-Agents package
• If you want to connect to a Unity editor, make sure the editor is running and the game scene is loaded
• If you want to connect to a Unity executable, make sure to provide the correct file path
• When using multiple MLPlay instances, make sure your Unity environment supports the requested number of agents
• Control modes:
  • manual: Player is controlled manually via keyboard/gamepad
  • hidden: Player is hidden (not visible in the game)
  • Python file path: Player is controlled by an MLPlay instance loaded from the specified file

Contributing

Pull requests and issues are welcome to improve this framework.

License

MIT