VisionRL

A Python package for visual Reinforcement Learning environments and tools.

VisionRL

A Python package for visual Reinforcement Learning environments and tools.

Design Philosophy

This library is designed as a Gymnasium-compatible extension, focused on computer vision, visualization, UI, and explainability for reinforcement learning. The design follows a layered and modular approach where each concern is clearly separated:

• Gymnasium handles agent contracts.
• VisionRL handles human understanding.
• Core logic is minimal and stable.
• Features are added via wrappers, not rewrites.
• UI, CV, and explainability are optional layers.

Project Structure

🔹 vision_rl/ (Root Package)

The main Python package. Everything inside is part of the public or internal API.

🔹 vision_rl/core/ – Core Abstractions

Defines the foundational building blocks.

• Extends Gymnasium’s Env.
• Adds support for visual observations, UI hooks, and debug callbacks.
• Key Files: visual_env.py (Base class), mixins.py (Reusable components).

🔹 vision_rl/wrappers/ – Environment Enhancements

The core extension mechanism. Most new features live here.

• Modifies Gym environments without changing their code.
• Handles frame stacking, visual conversions, heatmaps, and logging.

🔹 vision_rl/envs/ – Ready-to-Use Environments

Prebuilt environments for demonstration and best practices.

• Examples: VisualFrozenLake, VisualCartPole.

🔹 vision_rl/ui/ – Human-Facing Interfaces

Tools for visualization and interaction. Optional for training, critical for demos.

• Renders steps, plots metrics, and handles dashboards.

🔹 vision_rl/monitors/ – Training Analysis Tools

Tools to analyze agent behavior over time.

• Track rewards, action distributions, and learning progress.

🔹 vision_rl/utils/ – Shared Utilities

Helper functions for CV, rendering, and image processing. No environment logic lives here.

🔹 vision_rl/register.py – Gym Integration

Connects environments to Gymnasium’s registry (e.g., gym.make("VisualFrozenLake-v0")).