luckyrobots 0.1.80

Robotics-AI Training in Hyperrealistic Game Environments

Infinite synthetic data generation for embodied AI

https://github.com/user-attachments/assets/0ab2953d-b188-4af7-a225-71decdd2378c

Lucky Robots

Hyperrealistic robotics simulation framework with Python API for embodied AI training and testing.

Quick Start

1. Download LuckyEngine from our releases page and set the path:

   # Set environment variable (choose one method):
   
   # Method 1: Set LUCKYENGINE_PATH directly to the executable
   export LUCKYENGINE_PATH=/path/to/LuckyEngine      # Linux/Mac
   export LUCKYENGINE_PATH=/path/to/LuckyEngine.exe  # Windows
   
   # Method 2: Set LUCKYENGINE_HOME to the directory containing the executable
   export LUCKYENGINE_HOME=/path/to/luckyengine/directory
   

2. Install

   pip install luckyrobots
   

3. Run Example

   git clone https://github.com/luckyrobots/luckyrobots.git
   cd luckyrobots/examples
   python controller.py --skip-launch  # If LuckyEngine is already running
   

Basic Usage

Low-level client (direct gRPC)

from luckyrobots import LuckyEngineClient

client = LuckyEngineClient(host="127.0.0.1", port=50051, robot_name="unitreego2")
client.wait_for_server()

# RL step: send action, get observation
obs = client.step(actions=[0.0] * 12)
print(f"Observation: {obs.observation[:5]}...")

# Or separately:
client.send_control(controls=[0.1, 0.2, -0.1, ...])
obs = client.get_observation()
joints = client.get_joint_state()

High-level session (manages engine lifecycle)

from luckyrobots import Session

with Session() as session:
    session.start(scene="velocity", robot="unitreego2", task="locomotion")
    obs = session.step(actions=[0.0] * 12)
    obs = session.reset()

API Overview

Core Classes

LuckyEngineClient - Low-level gRPC client

• wait_for_server(timeout) - Wait for LuckyEngine connection
• step(actions) - Send actions + physics step + get observation (single RPC)
• get_observation() - Get RL observation vector
• get_joint_state() - Get joint positions/velocities
• send_control(controls) - Send actuator commands
• get_agent_schema() - Get observation/action names and sizes
• reset_agent() - Reset agent state
• set_simulation_mode(mode) - Set timing: "fast", "realtime", "deterministic"
• benchmark(duration, method) - Benchmark RPC latency

Session - Managed session (launches + connects to LuckyEngine)

• start(scene, robot, task) - Launch engine and connect
• connect(robot=) - Connect to already-running engine
• step(actions) - RL step
• reset() - Reset agent
• close() - Disconnect and stop engine

Models

from luckyrobots import ObservationResponse

# ObservationResponse - returned by step() and get_observation()
obs.observation      # List[float] - flat RL observation vector
obs.actions          # List[float] - last applied actions
obs.timestamp_ms     # int - wall-clock timestamp
obs.frame_number     # int - monotonic counter
obs["name"]          # Named access (if schema fetched)
obs.to_dict()        # Convert to name->value dict

System Identification (optional)

Calibrate MuJoCo model parameters to match real robot behavior.

pip install luckyrobots[sysid]

CLI

# Collect trajectory data from the engine
luckyrobots sysid collect --robot unitreego2 --signal chirp --duration 15 -o traj.npz

# Identify model parameters
luckyrobots sysid identify traj.npz -m go2.xml --preset go2:motor -o result.json

# Apply calibrated parameters to create a new model
luckyrobots sysid apply result.json -m go2.xml -o go2_calibrated.xml

# List available parameter presets
luckyrobots sysid presets

Python API

from luckyrobots.sysid import identify, apply_params, TrajectoryData, load_preset, chirp

# Generate excitation signal
ctrl = chirp(duration=15.0, dt=0.02, amplitude=0.3, num_joints=12)

# Load recorded trajectory
traj = TrajectoryData.load("trajectory.npz")

# Identify parameters
specs = load_preset("go2", "motor")  # armature, damping, frictionloss per joint
result = identify("go2.xml", traj, specs)

# Apply to MuJoCo XML
apply_params("go2.xml", result, "go2_calibrated.xml")

Available Robots & Environments

Robots

• unitreego2: Unitree Go2 quadruped (12 joints)
• so100: 6-DOF manipulator with gripper
• stretch_v1: Mobile manipulator

Scenes

• velocity: Velocity control training
• kitchen: Residential kitchen environment

Tasks

• locomotion: Walking/movement
• pickandplace: Object manipulation

Development

Setup with uv (recommended)

# Clone and enter repo
git clone https://github.com/luckyrobots/luckyrobots.git
cd luckyrobots

# Install uv if you haven't
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create venv and install deps
uv sync

# Run tests
uv run pytest

# Run example
uv run python examples/controller.py --skip-launch

Setup with pip

git clone https://github.com/luckyrobots/luckyrobots.git
cd luckyrobots
pip install -e ".[dev]"

Regenerating gRPC Stubs

The Python gRPC stubs are in src/luckyrobots/grpc/generated/ and are generated from protos in src/luckyrobots/grpc/proto/.

python -m grpc_tools.protoc \
  -I "src/luckyrobots/grpc/proto" \
  --python_out="src/luckyrobots/grpc/generated" \
  --grpc_python_out="src/luckyrobots/grpc/generated" \
  src/luckyrobots/grpc/proto/*.proto

Project Structure

src/luckyrobots/
├── client.py            # LuckyEngineClient — low-level gRPC client
├── session.py           # Session — managed engine lifecycle
├── debug.py             # Draw helpers (velocity arrows, lines)
├── sim_contract.py      # Simulation contract → protobuf builder
├── utils.py             # Shared utilities
├── models/              # Data classes
│   ├── observation.py   # ObservationResponse
│   └── benchmark.py     # BenchmarkResult, FPS
├── engine/              # Engine process management
├── grpc/                # gRPC internals
│   ├── generated/       # Protobuf stubs
│   └── proto/           # .proto files
├── config/              # Robot configurations (robots.yaml)
└── sysid/               # System identification (optional)
    ├── trajectory.py    # TrajectoryData (save/load recordings)
    ├── parameters.py    # ParamSpec, get/set MuJoCo params, presets
    ├── sysid.py         # identify() optimizer + SysIdResult
    ├── calibrate.py     # apply_params() to MuJoCo XML
    ├── collector.py     # Collector ABC + EngineCollector
    ├── excitation.py    # Signal generators (chirp, multisine, random_steps)
    └── cli.py           # luckyrobots sysid CLI

Contributing

1. Fork the repository
2. Create a feature branch
3. Make changes and add tests
4. Run uv run ruff check . and uv run ruff format .
5. Submit a pull request

Architecture

Lucky Robots uses gRPC for communication:

• LuckyEngine: Physics + rendering backend (Unreal Engine + MuJoCo)
• Python client: Connects via gRPC (default 127.0.0.1:50051)

gRPC Services

Service	Status	Description
MujocoService	✅ Working	Joint state, controls
AgentService	✅ Working	Observations, reset
SceneService	🚧 Placeholder	Scene inspection
TelemetryService	🚧 Placeholder	Telemetry streaming
CameraService	🚧 Placeholder	Camera frames
ViewportService	🚧 Placeholder	Viewport pixels

License

MIT License - see LICENSE file.

Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Discord: Community Server