Pixeler

A Python framework for building game automation bots that can see, understand, and interact with Windows applications in a human-like way.

How it works

Capture screenshots  →  Train a model  →  Run a GameBot  →  Write game modules
      (CLI)               (CLI/API)        (event loop)        (plugin API)

1. Capture — point the pixeler-capture tool at your game window and draw bounding boxes around the objects you care about (enemies, health bars, loot, UI elements). Frames are saved as a labeled dataset.
2. Train — run pixeler-train to fine-tune a YOLOv8 nano model on your dataset and export it as an ONNX file.
3. Detect — load the ONNX into a GameBot alongside color filters, templates, and OCR regions. Every frame, the ScreenAnalyzer runs all detectors and fires events.
4. React — write a GameModule that subscribes to events ("detection.enemy", "color.health_low", "ocr.health_bar") and calls mouse/keyboard actions in response.

Installation

pip install pixeler

Training support (YOLOv8 via Ultralytics) is an optional extra — skip it if you only need inference at runtime:

# Runtime only
pip install pixeler

# With training support
pip install "pixeler[train]"

Additional requirement: Tesseract OCR must be installed separately and its directory added to your system PATH before OCR functions will work.

Quickstart

Step 1 — Capture a dataset

pixeler-capture --game "MyGame" --dataset datasets/mygame --classes enemy,loot,health_bar

An OpenCV annotation window opens on top of a frozen game screenshot.

Step 2 — Train

pixeler-train --dataset datasets/mygame --out models/mygame.onnx --epochs 100

Exports the dataset into a YOLO-compatible structure, trains yolov8n, and writes the final model to models/mygame.onnx. Training artifacts (charts, confusion matrices, checkpoints) land in training_runs/mygame/.

Step 3 — Write a bot

# my_game_bot.py
from pathlib import Path
from src.pixeler.bot.game_bot import GameBot          # coming in phase 4
from src.pixeler.window.win32_window import Win32Window
from src.pixeler.vision.classifier import YOLOClassifier
from src.pixeler.vision.color import ColorFilter
from src.pixeler.math.rectangle import Rectangle
from my_game.combat_module import CombatModule

CLASSES = ["enemy", "loot", "health_bar"]

bot = GameBot(window=Win32Window("MyGame"))
bot.analyzer.add_yolo("detector", YOLOClassifier("models/mygame.onnx", CLASSES))
bot.analyzer.add_color("health_low", ColorFilter.from_rgb(220, 30, 30, hue_tol=10))
bot.analyzer.add_ocr("gold", Rectangle(10, 50, 120, 18), throttle_s=1.0)
bot.add_module(CombatModule())
bot.start()

Step 4 — Write a game module

# my_game/combat_module.py
from src.pixeler.modules.base_module import GameModule   # coming in phase 5
from src.pixeler.events.payloads import DetectionPayload, ColorRegionPayload

class CombatModule(GameModule):
    name = "combat"

    def on_register(self, bus, bot):
        self._bot = bot
        self.on("detection.enemy", self._attack)
        self.on("color.health_low", self._eat_food)

    def _attack(self, event):
        payload: DetectionPayload = event.data
        self.log(f"Enemy at {payload.center}")
        from src.pixeler.input.mouse import move_and_right_click
        move_and_right_click(*payload.center)

    def _eat_food(self, event):
        from src.pixeler.input.keyboard import press
        press("1")

Feature Overview

Vision

Training

Event System

Input

Window

Math & Geometry

Development Setup

git clone https://github.com/klobbix/Pixeler
cd Pixeler

# Install runtime dependencies
uv sync

# Install with training support
uv sync --extra train

# Run examples
uv run python examples/example_bot.py

License

MIT — see LICENSE.

Acknowledgments

• Ultralytics YOLOv8 — object detection training
• pytesseract — OCR
• OpenCV — computer vision
• PyAutoGUI — input simulation
• WindMouse algorithm — Jack Tasia

Contact

klobbix@gmail.com · GitHub Issues