NVIDIA Isaac Capture

Overview

Isaac Capture provides shared infrastructure for collecting human and robot demonstration data. It brings together two components:

• Isaac Teleop supports robot-in-the-loop capture through teleoperation, including Teleop Session and Remote Assist.
• Ego Pipeline supports human and egocentric data capture through Ego Capture and Ego Post Processing.

Both components use a shared core of device plugins, retargeters, and data utilities for recording, replaying, and converting data. Isaac Teleop remains a component of Isaac Capture, with its existing functionality preserved.

Key Features

• Unified stack for sim & real teleoperation
• Standardized device interface
• Flexible retargeting framework

Isaac Teleop

• Currently supported use cases
  • Use XR headsets for gripper / tri-finger hand manipulation
  • Use XR headsets with gloves for dex-hand manipulation
  • Seated full body loco-manipulation (Homie)
  • Tracking based full body loco-manipulation with GR00T-WholeBodyControl (SONIC)
• Upcoming use cases
  • Teleoperate using only non-XR devices (e.g. gamepad, Gello, haply, etc.)
  • Teleoperate cloud based robotics simulations
  • Remote teleoperation with camera streaming to the desktop
  • Remote teleoperation with immersive camera streaming to XR headsets

Ego Pipeline

• Capture egocentric data without a robot in the loop.

Quick Start

Documentation

Our documentation page provides everything you need to get started, including detailed tutorials and step-by-step guides. Follow these links to learn more:

• Architecture
• Quick installation steps
• How to build from source

Install & Run Isaac Lab

The latest NVIDIA Isaac Lab release, 3.0.0 EA, includes Isaac Capture, previously released as Isaac Teleop.

To get started, please refer to Isaac Lab's Installation guide for more details. Then follow the CloudXR teleoperation in Isaac Lab to get started with Teleoperation in Sim.