octorun 0.3.0

A command-line tool for distributed parallel execution across multiple GPUs

🐙 OctoRun

Distributed Parallel Execution Made Simple

A powerful command-line tool for running Python scripts across multiple GPUs with intelligent task management and monitoring

---

📋 Overview

OctoRun is designed to help you run computationally intensive Python scripts across multiple GPUs efficiently. It automatically manages GPU allocation, chunks your workload, handles failures with retry mechanisms, and provides comprehensive monitoring and logging.

✨ Key Features

• 🔍 Automatic GPU Detection: Automatically detects and utilizes available GPUs
• 🧩 Intelligent Chunk Management: Divides work into chunks and distributes across GPUs
• 🔄 Failure Recovery: Automatic retry mechanism for failed chunks
• 📊 Comprehensive Logging: Detailed logging for monitoring and debugging
• ⚙️ Flexible Configuration: JSON-based configuration with CLI overrides
• 🎯 Kwargs Support: Pass custom arguments to your scripts via config or CLI
• 💾 Memory Monitoring: Monitor GPU memory usage and thresholds
• 🔒 Lock Management: Prevent duplicate processing of chunks

🚀 Installation

You can install OctoRun using pip or uv.

Via pip

pip install octorun

Via uv

# Install globally
uv tool install octorun

# Install in your project
uv add octorun

Optional extras

• Benchmark tooling: pip install "octorun[benchmark]" (installs PyTorch with CUDA support)

⚡ Quick Start

1. Create Configuration:

   octorun save_config --script ./your_script.py
   

2. Run Your Script:

   octorun run
   

3. Monitor GPUs:

   octorun list_gpus -d
   

🎮 Commands

run (r)

Run your script with the specified configuration.

octorun run --config config.json [--kwargs '{"key": "value"}']

save_config (s)

Generate a default configuration file.

octorun save_config --script ./your_script.py

list_gpus (l)

List available GPUs and their current usage.

octorun list_gpus [--detailed]

The detailed flag provides a more comprehensive view of GPU stats, including memory usage, temperature, and running processes.

benchmark (b)

Run a benchmark to determine the optimal number of parallel processes for your GPUs.

octorun benchmark

This command runs a series of tests to help you configure the gpus parameter in your config.json for the best performance. Requires the optional benchmark extra (pip install "octorun[benchmark]") so PyTorch is available.

⚙️ Configuration

OctoRun uses a config.json file for configuration. You can generate a default one with octorun save_config.

Option	Description	Default
script_path	Path to your Python script	-
gpus	"auto" or list of GPU IDs	"auto"
total_chunks	Number of chunks to divide work into	128
log_dir	Directory for log files	"./logs"
chunk_lock_dir	Directory for chunk lock files	"./logs/locks"
monitor_interval	Monitoring interval in seconds	60
restart_failed	Whether to restart failed processes	false
max_retries	Maximum retries for failed chunks	3
memory_threshold	Memory threshold percentage	90
kwargs	Custom arguments to pass to your script	{}

🎯 Using Kwargs

You can pass custom arguments to your script via the kwargs object in your config.json or directly through the CLI.

CLI kwargs will override config file kwargs.

octorun run --kwargs '{"batch_size": 128, "learning_rate": 0.005}'

🔧 Script Implementation

Your script must accept the following arguments:

• --gpu_id: GPU device ID (int)
• --chunk_id: Current chunk number (int)
• --total_chunks: Total number of chunks (int)

Here is an example of how to structure your script:

import argparse
import torch

def main():
    parser = argparse.ArgumentParser()
    
    # Required OctoRun arguments
    parser.add_argument('--gpu_id', type=int, required=True)
    parser.add_argument('--chunk_id', type=int, required=True)
    parser.add_argument('--total_chunks', type=int, required=True)
    
    # Your custom arguments
    parser.add_argument('--batch_size', type=int, default=32)
    parser.add_argument('--learning_rate', type=float, default=0.001)
    parser.add_argument('--model_type', type=str, default='default')
    parser.add_argument('--epochs', type=int, default=1)
    parser.add_argument('--output_dir', type=str, default='./output')
    
    args = parser.parse_args()
    
    # Set the GPU device
    if torch.cuda.is_available():
        torch.cuda.set_device(args.gpu_id)
        print(f"Using GPU {args.gpu_id}")
    
    print(f"Processing chunk {args.chunk_id}/{args.total_chunks}")
    
    # Your logic here

if __name__ == "__main__":
    main()

🤝 Contributing

Contributions are welcome! Please fork the repository, create a feature branch, and submit a pull request.

📄 License

This project is licensed under the MIT License.