luckyrobots 0.1.69

Robotics-AI Training in Hyperrealistic Game Environments

Infinite synthetic data generation for embodied AI

https://github.com/user-attachments/assets/0ab2953d-b188-4af7-a225-71decdd2378c

Lucky Robots

Hyperrealistic robotics simulation framework with Python API for embodied AI training and testing.

Quick Start

1. Download LuckyEngine from our releases page and set the path:

   # Set environment variable (choose one method):
   
   # Method 1: Set LUCKYENGINE_PATH directly to the executable
   export LUCKYENGINE_PATH=/path/to/LuckyEngine      # Linux/Mac
   export LUCKYENGINE_PATH=/path/to/LuckyEngine.exe  # Windows
   
   # Method 2: Set LUCKYENGINE_HOME to the directory containing the executable
   export LUCKYENGINE_HOME=/path/to/luckyengine/directory
   

2. Install

   pip install luckyrobots
   

3. Run Example

   git clone https://github.com/luckyrobots/luckyrobots.git
   cd luckyrobots/examples
   python controller.py --skip-launch  # If LuckyEngine is already running
   

Basic Usage

from luckyrobots import LuckyEngineClient

# Connect to LuckyEngine
client = LuckyEngineClient(
    host="127.0.0.1",
    port=50051,
    robot_name="unitreego1",
)
client.wait_for_server()

# Optional: Fetch schema for named observation access
client.fetch_schema()

# Get RL observation
obs = client.get_observation()
print(f"Observation: {obs.observation[:5]}...")  # Flat vector for RL
print(f"Timestamp: {obs.timestamp_ms}")

# Named access (if schema fetched)
# obs["proj_grav_x"]  # Access by name
# obs.to_dict()       # Convert to dict

# Send controls
client.send_control(controls=[0.1, 0.2, -0.1, ...])

# Get joint state (separate from RL observation)
joints = client.get_joint_state()
print(f"Positions: {joints.positions}")
print(f"Velocities: {joints.velocities}")

API Overview

Core Classes

LuckyEngineClient - Low-level gRPC client

• wait_for_server(timeout) - Wait for LuckyEngine connection
• get_observation() - Get RL observation vector
• get_joint_state() - Get joint positions/velocities
• send_control(controls) - Send actuator commands
• get_agent_schema() - Get observation/action names and sizes
• reset_agent() - Reset agent state

LuckyRobots - High-level wrapper (launches LuckyEngine)

• start(scene, robot, task) - Launch and connect
• get_observation() - Get observation
• step(controls) - Send controls and get next observation

Models

from luckyrobots import ObservationResponse, StateSnapshot

# ObservationResponse - returned by get_observation()
obs.observation      # List[float] - flat RL observation vector
obs.actions          # List[float] - last applied actions
obs.timestamp_ms     # int - wall-clock timestamp
obs.frame_number     # int - monotonic counter
obs["name"]          # Named access (if schema fetched)
obs.to_dict()        # Convert to name->value dict

Available Robots & Environments

Robots

• unitreego1: Quadruped robot
• so100: 6-DOF manipulator with gripper
• stretch_v1: Mobile manipulator

Scenes

• velocity: Velocity control training
• kitchen: Residential kitchen environment

Tasks

• locomotion: Walking/movement
• pickandplace: Object manipulation

Development

Setup with uv (recommended)

# Clone and enter repo
git clone https://github.com/luckyrobots/luckyrobots.git
cd luckyrobots

# Install uv if you haven't
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create venv and install deps
uv sync

# Run tests
uv run pytest

# Run example
uv run python examples/controller.py --skip-launch

Setup with pip

git clone https://github.com/luckyrobots/luckyrobots.git
cd luckyrobots
pip install -e ".[dev]"

Regenerating gRPC Stubs

The Python gRPC stubs are in src/luckyrobots/grpc/generated/ and are generated from protos in src/luckyrobots/grpc/proto/.

python -m grpc_tools.protoc \
  -I "src/luckyrobots/grpc/proto" \
  --python_out="src/luckyrobots/grpc/generated" \
  --grpc_python_out="src/luckyrobots/grpc/generated" \
  src/luckyrobots/grpc/proto/*.proto

Project Structure

src/luckyrobots/
├── client.py            # LuckyEngineClient (main API)
├── luckyrobots.py       # LuckyRobots high-level wrapper
├── models/              # Pydantic models
│   ├── observation.py   # ObservationResponse, StateSnapshot
│   └── camera.py        # CameraData, CameraShape
├── engine/              # Engine management
├── grpc/                # gRPC internals
│   ├── generated/       # Protobuf stubs
│   └── proto/           # .proto files
└── config/              # Robot configurations

Contributing

1. Fork the repository
2. Create a feature branch
3. Make changes and add tests
4. Run uv run ruff check . and uv run ruff format .
5. Submit a pull request

Architecture

Lucky Robots uses gRPC for communication:

• LuckyEngine: Physics + rendering backend (Unreal Engine + MuJoCo)
• Python client: Connects via gRPC (default 127.0.0.1:50051)

gRPC Services

Service	Status	Description
MujocoService	✅ Working	Joint state, controls
AgentService	✅ Working	Observations, reset
SceneService	🚧 Placeholder	Scene inspection
TelemetryService	🚧 Placeholder	Telemetry streaming
CameraService	🚧 Placeholder	Camera frames
ViewportService	🚧 Placeholder	Viewport pixels

License

MIT License - see LICENSE file.

Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Discord: Community Server