luckyrobots 0.1.60

Robotics-AI Training in Hyperrealistic Game Environments

Infinite synthetic data generation for embodied AI

https://github.com/user-attachments/assets/0ab2953d-b188-4af7-a225-71decdd2378c

Lucky Robots

Hyperrealistic robotics simulation framework with Python API for embodied AI training and testing.

Quick Start

1. Create conda environment (recommended)

   conda create -n luckyrobots python
   conda activate luckyrobots
   

2. Install

   pip install luckyrobots
   

3. Run Example

   git clone https://github.com/luckyrobots/luckyrobots.git
   cd luckyrobots/examples
   python controller.py
   

Basic Usage

from luckyrobots import LuckyRobots, Node
import numpy as np

# Create controller node
class RobotController(Node):
    async def control_loop(self):
        # Reset environment
        reset_response = await self.reset_client.call(Reset.Request())
        
        # Send actions
        actuator_values = np.array([0.1, 0.2, -0.1, 0.0, 0.5, 1.0])
        step_response = await self.step_client.call(Step.Request(actuator_values=actuator_values))
        
        # Access observations
        observation = step_response.observation
        joint_states = observation.observation_state
        camera_data = observation.observation_cameras

# Start simulation
luckyrobots = LuckyRobots()
controller = RobotController()
luckyrobots.register_node(controller)
luckyrobots.start(scene="kitchen", robot="so100", task="pickandplace")

Available Robots & Environments

Robots

• so100: 6-DOF manipulator with gripper
• stretch_v1: Mobile manipulator
• dji300: Quadcopter drone

Scenes

• kitchen: Residential kitchen environment
• loft: Open floor plan apartment
• drone_flight: Outdoor flight area

Tasks

• pickandplace: Object manipulation
• navigation: Path planning and movement

API Reference

Core Classes

LuckyRobots: Main simulation manager

• start(scene, robot, task, observation_type): Initialize simulation
• register_node(node): Add controller node
• spin(): Run main loop

Node: Base class for robot controllers

• create_client(service_type, service_name): Create service client
• create_service(service_type, service_name, handler): Create service server

Services

Reset: Reset robot to initial state

request = Reset.Request(seed=42, options={})
response = await reset_client.call(request)

Step: Send action and get observation

request = Step.Request(actuator_values=[0.1, 0.2, -0.1])
response = await step_client.call(request)

Observations

Access sensor data from step responses:

# Joint positions and velocities
joint_states = response.observation.observation_state

# Camera images (RGB + depth)
for camera in response.observation.observation_cameras:
    image = camera.image_data  # numpy array
    name = camera.camera_name  # "head_cam", "hand_cam", etc.

Command Line Interface

# Basic usage
python controller.py --robot so100 --scene kitchen --task pickandplace

# With camera display
python controller.py --show-camera --rate 30

# Custom host/port
python controller.py --host 192.168.1.100 --port 3001

Configuration

Robot configurations are defined in src/luckyrobots/config/robots.yaml:

so100:
  action_space:
    actuator_names: [shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper]
    actuator_limits:
      - name: shoulder_pan
        lower: -2.2
        upper: 2.2
  available_scenes: [kitchen]
  available_tasks: [pickandplace]

Architecture

Lucky Robots uses a distributed node architecture:

• Manager Node: Central message routing
• LuckyRobots Node: Simulation interface
• Controller Nodes: User-defined robot controllers
• WebSocket Transport: Inter-node communication
• Lucky World: Physics simulation backend

Development

Setup Development Environment

git clone https://github.com/luckyrobots/luckyrobots.git
cd luckyrobots
pip install -e .

Library Development Mode

python -m luckyrobots --lr-library-dev

Creates a symlink for local development without reinstalling.

Contributing

1. Fork the repository
2. Create a feature branch
3. Make changes and add tests
4. Submit a pull request

License

MIT License - see LICENSE file.

Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Discord: Community Server