TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation

Shivin Dass¹, Wensi Ai², Yuqian Jiang¹, Samik Singh¹, Jiaheng Hu², Ruohan Zhang¹, Peter Stone^1,3, Ben Abbatematteo¹, Roberto Martín-Martín¹

¹The University of Texas at Austin, ²Stanford University, ³Sony AI

[Paper], [Project Website]

TeleMoMa is a teleoperation toolkit that enables intuitive teleoperation of high-DoF mobile manipulators. We provide a common pipeline that supports several teleoperation interfaces such as vision, VR, mobile phones and more, and enables mixing-and-matching between them. While we provide code for several mobile manipulators in sim and real, the versatility of TeleMoMa allows easy plug-and-play teleoperation of any mobile manipulator in general.

Setup

System Requirements

• Ubuntu 20.04
• Python 3.9+

Other system configurations may work, but we haven’t tested them extensively and may only be able to provide limited support.

Note: TeleMoMa can work with higher versions of python but if iGibson sim environment is required then please use python3.9.

Installation

git clone https://github.com/UT-Austin-RobIn/telemoma.git
cd telemoma
conda create --name telemoma python==3.9
conda activate telemoma
pip install -e .

Install iGibson (Optional)

We provide a demo script to test different teleoperation modes in the iGibson sim environment. iGibson can be installed by running,

(telemoma) > pip install igibson

In case of installation or runtime issues with iGibson, please checkout their exhaustive documentation and github.

Usage

Quickstart

To verify your installation the following code can be used to run TeleMoMa in a sandbox environment in iGibson.

(telemoma) > python telemoma/demo_igibson.py --robot tiago --teleop_config telemoma/configs/only_keyboard.py 

--teleop_config defines the input modality to be used for controlling each part of the robot. Some example configurations are provided in telemoma/configs folder.

Make sure the input hardware is correctly setup by following the instructions provided in Human Interface.

Human Interface

Hardware specific setup instructions for devices compatible with TeleMoMa are provided here.

Robot Interface

The details on the sim environment installations, real robot codebase and instructions on setting up TeleMoMa on your own robot are provided here.

Citation

@article{dass2024telemoma,
  title={TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation},
  author={Dass, Shivin and Ai, Wensi and Jiang, Yuqian and Singh, Samik and Hu, Jiaheng and Zhang, Ruohan and Stone, Peter and Abbatematteo, Ben and Martín-Martín, Roberto},
  journal={arXiv preprint arXiv:2403.07869},
  year={2024}
}

Acknowledgements: Mentioned here