NWO Robotics CS API

Computer Vision & Control Systems API for NWO Robotics with HOI-PAGE Integration

NWO Robotics CS API provides advanced computer vision, motion planning, and human-robot interaction capabilities for the NWO Robotics ecosystem. Built with integrated HOI-PAGE (Human-Object Interaction) technology for zero-shot object manipulation and natural motion generation.

🚀 Features

Core Capabilities

• 🔍 Affordance Detection - Understand object parts and their uses
• 🤖 Motion Generation - Human-like robot motion planning
• 👥 Human Intent Prediction - Predict and respond to human actions
• 🎯 Zero-Shot Manipulation - Handle novel objects without training
• 🧠 Embodied AI - Real-world robot learning and adaptation
• 🔗 NWO Integration - Seamless connection to NWO Robotics API

HOI-PAGE Integration

• Part-based object understanding
• Natural human-object interaction generation
• Collaborative task planning
• Sim-to-real transfer

📦 Installation

From PyPI (Recommended)

pip install nwo-robotics-cs

From Source

git clone https://github.com/RedCiprianPater/nwo-robotics-cs.git
cd nwo-robotics-cs
pip install -e .

With Optional Dependencies

# Full installation with all features
pip install nwo-robotics-cs[full]

# Development installation
pip install nwo-robotics-cs[dev]

# GPU support (CUDA)
pip install nwo-robotics-cs[gpu]

🔧 Quick Start

1. Configure API Access

# Set environment variables
export NWO_API_URL="https://nwo.capital/api"
export NWO_WALLET_ADDRESS="0xYourWalletAddress"
export NWO_PRIVATE_KEY="0xYourPrivateKey"

# Or use CLI configuration
nwo-cs config --api-url https://nwo.capital/api --wallet 0x...

2. Basic Usage

from nwo_robotics_cs import NWORoboticsClient, HOIPAGEIntegration

# Initialize client
client = NWORoboticsClient(
    api_url="https://nwo.capital/api",
    wallet_address="0x..."
)

# Analyze object affordances
object_scan = client.camera.capture_3d()
affordances = client.hoi_page.get_affordances(object_scan)

print(f"Object parts: {[part.name for part in affordances.parts]}")
# Output: ['handle', 'body', 'spout']

# Generate manipulation motion
motion = client.hoi_page.generate_motion(
    object_id="mug_001",
    task="pour_water",
    style="human_like"
)

# Execute with NWO robot
client.robot.execute_motion(motion, auth_check=True)

3. CLI Usage

# Analyze object
nwo-cs analyze --image cup.jpg --output affordances.json

# Generate motion
nwo-cs generate-motion --object mug.obj --task "pour_water" --output motion.pkl

# Execute on robot
nwo-cs execute --motion motion.pkl --robot-id "nwo-bot-001"

# Predict human intent
nwo-cs predict-intent --video stream.mp4 --context kitchen

📚 API Reference

NWORoboticsClient

Main client for interacting with NWO Robotics API.

from nwo_robotics_cs import NWORoboticsClient

client = NWORoboticsClient(
    api_url="https://nwo.capital/api",
    wallet_address="0x...",
    private_key="0x...",  # Optional
    contract_address="0x..."  # Optional
)

HOIPAGEIntegration

Human-Object Interaction generation using HOI-PAGE.

from nwo_robotics_cs import HOIPAGEIntegration

hoi = HOIPAGEIntegration(model="hoi-page-v1")

# Detect affordances
affordances = hoi.get_affordances(
    point_cloud=point_cloud_data,
    image=rgb_image
)

# Generate motion
motion = hoi.generate_motion(
    object_geometry=mesh_data,
    task="grasp_and_lift",
    constraints=["no_collision", "minimum_force"]
)

# Predict human intent
intent = hoi.predict_human_intent(
    video_frames=frames,
    context="collaborative_assembly"
)

AffordanceDetector

Detect object parts and their functional uses.

from nwo_robotics_cs import AffordanceDetector

detector = AffordanceDetector()

# Analyze object
result = detector.analyze(
    rgb_image=image,
    depth_map=depth,
    point_cloud=point_cloud
)

for part in result.parts:
    print(f"{part.name}: {part.affordances} (confidence: {part.confidence})")

MotionPlanner

Generate and optimize robot motions.

from nwo_robotics_cs import MotionPlanner

planner = MotionPlanner(robot_model="ur5e")

# Plan motion
trajectory = planner.plan(
    start_pose=current_pose,
    goal_pose=target_pose,
    obstacles=obstacle_list,
    style="human_like"
)

# Optimize for smoothness
optimized = planner.optimize(trajectory, criterion="jerk_minimization")

🔌 NWO Robotics Integration

Authentication

from nwo_robotics_cs import NWORoboticsClient

# Web3 wallet authentication
client = NWORoboticsClient(
    api_url="https://nwo.capital/api",
    wallet_address="0x...",
    private_key="0x..."
)

# Verify connection
health = client.health_check()
print(f"API Status: {health.status}")

Signal Analysis Integration

# Connect to NWO Signal Spectrum
signals = client.signal_spectrum.get_signals(
    freq_min=433000000,
    freq_max=434000000
)

# Share detected anomalies
for signal in signals:
    if signal.classification == "unknown":
        client.signal_spectrum.share_signal({
            "frequency_hz": signal.frequency_hz,
            "modulation": signal.modulation,
            "confidence": signal.confidence
        })

Agent Coordination

# Join agent network
client.agent_network.join(wallet_address="0x...")

# Submit task for consensus
task = {
    "type": "object_manipulation",
    "object_id": "obj_001",
    "proposed_action": "grasp_handle"
}

consensus = client.agent_network.submit_task(task)
print(f"Consensus: {consensus.action} (confidence: {consensus.confidence})")

🎯 Advanced Examples

Example 1: Collaborative Assembly

from nwo_robotics_cs import NWORoboticsClient, HOIPAGEIntegration
import asyncio

async def collaborative_assembly():
    client = NWORoboticsClient()
    hoi = HOIPAGEIntegration()
    
    # Monitor human worker
    human_pose = client.vision.get_human_pose()
    intent = hoi.predict_human_intent(human_pose)
    
    if intent.action == "reach_for_screw":
        # Robot prepares screwdriver
        screw = client.vision.detect_object("screw")
        affordances = hoi.get_affordances(screw)
        
        motion = hoi.generate_motion(
            object_geometry=screw.mesh,
            task="pick_up",
            hand_preference="right"
        )
        
        # Execute with safety checks
        await client.robot.execute_motion(
            motion,
            collision_check=True,
            speed=0.5  # Slow for collaboration
        )
        
        # Hand over to human
        await client.robot.move_to("handover_position")
        print("Screw ready for human")

asyncio.run(collaborative_assembly())

Example 2: Zero-Shot Kitchen Task

from nwo_robotics_cs import NWORoboticsClient

client = NWORoboticsClient()

# Encounter novel object (never seen before)
unknown_object = client.vision.scan_object()

# HOI-PAGE analyzes
analysis = client.hoi_page.analyze(unknown_object)

print(f"Detected: {analysis.object_type}")
print(f"Parts: {[p.name for p in analysis.parts]}")

# Task: "Make coffee"
if analysis.object_type == "coffee_maker":
    # Generate motion without prior training
    steps = [
        {"action": "open_lid", "part": "lid"},
        {"action": "pour_water", "part": "reservoir"},
        {"action": "add_coffee", "part": "filter_compartment"},
        {"action": "close_lid", "part": "lid"},
        {"action": "press_button", "part": "power_button"}
    ]
    
    for step in steps:
        motion = client.hoi_page.generate_motion(
            object_geometry=unknown_object.mesh,
            task=step["action"],
            target_part=step["part"]
        )
        client.robot.execute_motion(motion)

Example 3: Safety-Aware Motion

from nwo_robotics_cs import MotionPlanner, SafetyMonitor

planner = MotionPlanner()
monitor = SafetyMonitor()

# Plan motion in human-occupied space
human_position = client.vision.get_human_position()

motion = planner.plan(
    goal="place_object_on_table",
    constraints=[
        "maintain_1m_distance_from_human",
        "avoid_human_workspace",
        "visible_to_human"
    ],
    human_position=human_position
)

# Monitor during execution
for waypoint in motion.waypoints:
    if monitor.check_safety(waypoint):
        client.robot.move_to(waypoint)
    else:
        client.robot.emergency_stop()
        monitor.alert("Safety violation detected")

🛠️ CLI Reference

Global Options

nwo-cs [OPTIONS] COMMAND [ARGS]

Options:
  --config PATH          Config file path
  --api-url TEXT         NWO API URL
  --wallet TEXT          Wallet address
  --verbose              Enable verbose output
  --help                 Show help message

Commands

config

Configure CLI settings

nwo-cs config --api-url https://nwo.capital/api --wallet 0x...

analyze

Analyze object affordances

nwo-cs analyze --image object.jpg --output analysis.json
nwo-cs analyze --point-cloud object.ply --visualize

generate-motion

Generate robot motion

nwo-cs generate-motion --object mug.obj --task "pour_water" --output motion.pkl
nwo-cs generate-motion --object-id obj_123 --task "grasp" --style human_like

execute

Execute motion on robot

nwo-cs execute --motion motion.pkl --robot-id bot_001
nwo-cs execute --trajectory traj.json --speed 0.5 --collision-check

predict-intent

Predict human intent

nwo-cs predict-intent --video stream.mp4 --output intent.json
nwo-cs predict-intent --camera 0 --real-time

monitor

Monitor robot status

nwo-cs monitor --robot-id bot_001
nwo-cs monitor --all --format json

signal

Signal spectrum operations

nwo-cs signal scan --frequency 433.92e6 --duration 30
nwo-cs signal share --signal-id sig_001
nwo-cs signal consensus --signal-id sig_001 --classification voice

🔧 Configuration

Config File (~/.nwo_cs/config.yaml)

api:
  url: https://nwo.capital/api
  timeout: 30
  retry_count: 3

auth:
  wallet_address: "0x..."
  private_key: "0x..."  # Stored securely
  contract_address: "0x..."

robot:
  default_model: "ur5e"
  safety_limits:
    max_speed: 1.0
    max_force: 100
    workspace_bounds: [[-1, 1], [-1, 1], [0, 2]]

hoi_page:
  model: "hoi-page-v1"
  device: "cuda"  # or "cpu"
  batch_size: 4

logging:
  level: INFO
  file: ~/.nwo_cs/logs/nwo_cs.log

🧪 Testing

# Run all tests
pytest

# Run specific test file
pytest tests/test_hoi_page.py

# Run with coverage
pytest --cov=nwo_robotics_cs --cov-report=html

# Run integration tests
pytest tests/integration/ -v

📖 Documentation

• API Reference
• Tutorials
• Examples
• Changelog

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

📄 License

MIT License - see LICENSE file

🙏 Acknowledgments

• HOI-PAGE by Craig Lei Li
• NWO Robotics by NWO Capital
• Built on PyTorch, ROS2, and Open3D

📞 Support

• GitHub Issues: https://github.com/RedCiprianPater/nwo-robotics-cs/issues
• Discord: https://discord.gg/nwo
• Email: dev@nwo.capital

---

Built with 💚 for the NWO Robotics Network