rl-arena 0.1.0

A Python library for competitive reinforcement learning environments

RL Arena 🎮

A Python library for competitive reinforcement learning environments, similar to kaggle-environments but focused on multi-agent RL research and competitions.

🌟 Features

• Easy-to-use API: Familiar Gymnasium-style interface
• Competitive Multi-Agent: Built for head-to-head competition
• Extensible: Simple framework for adding new environments
• Well-tested: Comprehensive test suite with >80% coverage
• Type-safe: Full type hints for better development experience
• Reproducible: Deterministic environments with seed support
• Visualization System: Real-time rendering with Matplotlib
• Replay Recording: Save and replay matches in JSON format
• HTML5 Replays: Interactive browser-based match playback

📦 Installation

From PyPI (recommended)

pip install rl-arena

From Source

git clone https://github.com/rl-arena/rl-arena-env.git
cd rl-arena-env
pip install -e .

For Development

git clone https://github.com/rl-arena/rl-arena-env.git
cd rl-arena-env
pip install -e ".[dev]"

🚀 Quick Start (30 seconds!)

RL-Arena makes it easy to: Create environments → Train agents → Test → Submit

import rl_arena

# 1️⃣ Create environment
env = rl_arena.make("pong")

# 2️⃣ Train agent (DQN)
model = rl_arena.train_dqn("pong", total_timesteps=10000)

# 3️⃣ Test agent
agent = rl_arena.create_agent(model)
results = rl_arena.evaluate(agent, "pong", n_episodes=10)
print(f"Average reward: {results['mean_reward']:.2f}")

# 4️⃣ Create submission file
rl_arena.create_submission(
    agent,
    "my_submission.py",
    agent_name="MyAgent",
    author="your_name"
)

That's it! Submit my_submission.py to rl-arena-backend.

---

📖 Basic Usage

Environment Creation

import rl_arena

# Create environment
env = rl_arena.make("pong")

# Use built-in agents
agent1 = rl_arena.RandomAgent()
agent2 = rl_arena.RuleBasedAgent()

# Run a game
observations, info = env.reset(seed=42)
done = False

while not done:
    # Get actions from agents
    actions = [
        agent1.act(observations[0]),
        agent2.act(observations[1])
    ]
    
    # Step environment
    observations, rewards, terminated, truncated, info = env.step(actions)
    done = terminated or truncated
    
    # Render
    env.render()

print(f"Final scores: {info['scores']}")
env.close()

🎯 Available Environments

Environment	Players	Description	Complexity
pong	2	Classic Pong game	⭐ Easy

More environments coming soon! Contributions welcome!

🎮 Environment Details

Pong

Classic 2-player Pong game with customizable physics.

env = rl_arena.make("pong", configuration={
    "winning_score": 11,    # First to 11 wins
    "max_steps": 1000,      # Episode length limit
    "ball_speed": 0.02,     # Ball movement speed
    "paddle_height": 0.2,   # Paddle size
})

State Space: Ball position & velocity, paddle positions, scores
Action Space: Discrete(3) - UP, STAY, DOWN
Rewards: +1 for scoring, -1 for conceding

See environment documentation for more details.

📚 Documentation

• Quick Start: Get started in 5 minutes
• Environment Guide: Create custom environments
• API Reference: Complete API documentation
• Contributing: Contribution guidelines

🔨 Development

Running Tests

# Run all tests
pytest

# Run with coverage
pytest --cov=rl_arena tests/

# Run specific test file
pytest tests/test_core.py -v

Code Quality

# Format code
black rl_arena/ tests/ examples/

# Type checking
mypy rl_arena/

# Linting
flake8 rl_arena/ tests/ examples/

🤝 Contributing

We welcome contributions! Here's how you can help:

1. Add New Environments: See Environment Guide
2. Fix Bugs: Check open issues
3. Improve Documentation: Help make RL Arena more accessible
4. Add Examples: Share your agent implementations

See CONTRIBUTING.md for detailed guidelines.

Quick Contribution Workflow

# Fork and clone
git clone https://github.com/YOUR-USERNAME/rl-arena-env.git
cd rl-arena-env

# Create branch
git checkout -b feature/my-new-environment

# Make changes, add tests, update docs

# Run tests and linting
pytest
black rl_arena/ tests/
flake8 rl_arena/

# Commit and push
git commit -m "feat: Add new environment"
git push origin feature/my-new-environment

# Open Pull Request on GitHub

📖 Examples

Visualize Games

# Real-time visualization with Matplotlib
python examples/visualize_game.py

Record and Replay Matches

# Record a match to JSON
python examples/record_match.py

# Convert recording to HTML5 replay
python examples/replay_to_html.py

# Open recordings/pong_replay.html in your browser

Using Visualization in Code

from rl_arena.envs.pong.environment import PongEnvironment
from rl_arena.core.recorder import MatchRecorder

# Create environment with visualization
env = PongEnvironment()
env.reset()

# Enable state recording for replay
env.enable_state_recording(True)

# Run game with rendering
for _ in range(100):
    actions = [env.action_space.sample(), env.action_space.sample()]
    env.step(actions)
    env.render(mode='human')  # Show in Matplotlib window

# Save replay as HTML
history = env.get_state_history()
from rl_arena.utils.replay import replay_to_html
html = replay_to_html({'frames': history}, 'Pong', 'replay.html')

Record with MatchRecorder

from rl_arena.core.recorder import MatchRecorder

# Create recorder
recorder = MatchRecorder(metadata={'player1': 'Agent1', 'player2': 'Agent2'})
recorder.start_recording()

# Record each frame during gameplay
for step in range(100):
    state = env._get_render_state()
    recorder.record_frame(state, actions, rewards, info)

recorder.stop_recording()
recorder.save('match.json')

Random Agents

python examples/random_agent.py

Train a DQN Agent

pip install stable-baselines3[extra]
python examples/train_dqn_pong.py --train --timesteps 100000

Run Local Matches

python examples/run_local_match.py

Agent Submission Template

python examples/submission_template.py

🎓 Use Cases

• Research: Study multi-agent RL algorithms
• Education: Learn RL in competitive settings
• Competitions: Host RL tournaments and competitions
• Benchmarking: Compare agent performance across environments
• Fun: Build and battle AI agents!

🏗️ Project Structure

rl-arena-env/
├── rl_arena/              # Main package
│   ├── core/              # Base classes and interfaces
│   ├── envs/              # Environment implementations
│   │   ├── pong/          # Pong environment
│   │   └── __template__/  # Template for new environments
│   └── utils/             # Utilities (replay, logging, etc.)
├── tests/                 # Test suite
├── examples/              # Example scripts
├── docs/                  # Documentation
└── .github/               # CI/CD workflows

🙏 Acknowledgments

• Inspired by kaggle-environments
• Built on Gymnasium
• Thanks to all contributors

📬 Contact & Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions

⭐ Star History

If you find RL Arena useful, please consider starring the repository!

---

Made with ❤️ by the RL Arena community