MAPLE - A unified CLI daemon for evaluating robotics policies across diverse simulation environments

Why Maple?

Evaluating robotics policies—whether Vision-Language-Action (VLA) models, foundation models, imitation learning policies, or reinforcement learning agents—across different simulation environments is painful:

• Environment chaos: Every simulator has its own observation format, action space, and API quirks
• Dependency hell: MuJoCo, PyBullet, Isaac Gym, LIBERO—each with conflicting dependencies
• Integration tax: Each policy-environment combination requires custom glue code
• No standardization: Comparing policies across environments means rewriting evaluation scripts

Maple solves this with a daemon-based architecture that containerizes everything:

# Start the daemon
maple serve

# Pull and serve a policy
maple pull policy openvla:7b
maple serve policy openvla:7b

# Pull and serve an environment
maple pull env libero
maple serve env libero

# Run evaluation
maple eval openvla-7b-xxx libero-yyy --tasks libero_10 --seeds 0,1,2

That's it. No dependency conflicts. No custom scripts. Just results.

Features

• 🐳 Docker-First Architecture — Every policy and environment runs in its own container
• 🔌 Adapter System — Automatic translation between policy outputs and environment inputs
• 📊 Batch Evaluation — Run evaluations across multiple tasks, seeds, and configurations
• ⚙️ Flexible Configuration — YAML config files, environment variables, or CLI flags
• 🏥 Health Monitoring — Background health checks with auto-restart on failure
• 💾 Persistent State — SQLite-backed state storage for tracking history

Installation

pip install maple-robotics

Requirements

• Python 3.10+
• Docker with NVIDIA GPU support
• NVIDIA GPU with CUDA 12.1+

Build Docker Images

# Policy images
docker build -t maple/openvla:latest docker/openvla/
docker build -t maple/smolvla:latest docker/smolvla/

# Environment images
docker build -t maple/libero:latest docker/libero/

Quick Start

# 1. Start daemon
maple serve --detach

# 2. Pull and serve policy
maple pull policy openvla:7b
maple serve policy openvla:7b
# Output: Policy ID: openvla-7b-a1b2c3d4

# 3. Pull and serve environment
maple pull env libero
maple serve env libero
# Output: Env ID: libero-x1y2z3w4

# 4. Run evaluation
maple eval openvla-7b-a1b2c3d4 libero-x1y2z3w4 \
    --tasks libero_10 \
    --seeds 0,1,2 \
    --output results/

# 5. View results
maple report results/

Supported Policies and Environments

All the list of supported environments and policies can be Found - Policies & Environments Reference. Feel free to suggest any specific policy or environment you would like to be added in the repository.

Architecture

┌─────────────────────────────────────────────────────────┐
│                      maple CLI                          │
└───────────────────────┬─────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────────────┐
│                   Maple Daemon                          │
│  ┌───────────┐  ┌───────────┐  ┌─────────────────────┐  │
│  │ Policy    │  │ Env       │  │ Adapter             │  │
│  │ Backends  │  │ Backends  │  │ Registry            │  │
│  └─────┬─────┘  └─────┬─────┘  └─────────────────────┘  │
└────────┼──────────────┼─────────────────────────────────┘
         │              │
         ▼              ▼
┌─────────────────┐  ┌─────────────────┐
│ Policy Container│  │ Env Container   │
│ (Docker + GPU)  │  │ (Docker + X11)  │
└─────────────────┘  └─────────────────┘

Configuration

Create ~/.maple/config.yaml:

daemon:
  port: 8000

policy:
  default_device: cuda:0
  model_kwargs: {}
  model_load_kwargs: {}

containers:
  memory_limit: 32g
  startup_timeout: 300

eval:
  max_steps: 300
  save_video: false

Or use environment variables:

MAPLE_DEVICE=cuda:1 maple serve

Documentation

Full documentation: maple-robotics.readthedocs.io

• Installation Guide
• Quick Start
• CLI Reference
• API Reference

License

MIT License - see LICENSE for details.