videosdk-teleop

Drive a robot arm over the internet, and know exactly what the operator was looking at when they moved it.

A WebRTC transport for robot teleoperation, with frame-accurate observation/action correlation, a safety envelope that cannot be bypassed, and a recorder that turns every session into training data.

Install

pip install videosdk-teleop               # transport + cameras, ready to run
pip install "videosdk-teleop[lerobot]"    # + the lerobot adapters
pip install "videosdk-teleop[ros2]"       # + the ROS 2 nodes

Your robot, in three methods

Everything else is inherited: lease negotiation, clamping, the watchdog, the e-stop latch, correlation, recording.

from videosdk_teleop import Follower, SafetyConfig, single_group_descriptor

class MyArm(Follower):
    def descriptor(self):            return single_group_descriptor(...)
    def read_joints(self):           return {"shoulder": 12.4, "elbow": -3.1}
    def write_joints(self, joints):  ...        # RAISE on failure

MyArm("abcd-efgh-ijkl", safety=SafetyConfig(slew=0.75)).run(hz=200)

Already on lerobot? Wrap the device instead of subclassing:

from videosdk_teleop.adapters.lerobot import (SO101FollowerAdapter,
                                              SO101LeaderAdapter)

follower = SO101FollowerAdapter("abcd-efgh-ijkl", robot=arm, token=TOKEN)

Run it locally, no robot needed

Runnable examples live in videosdk-teleops-examples:

git clone https://github.com/videosdk-live/videosdk-teleops-examples
cd videosdk-teleops-examples

python quickstart/remote_teleop.py --ticks 100   # full chain, in-process
python quickstart/my_arm.py                      # port your own arm
python quickstart/protocol_demo.py               # the correlation model, live

remote_teleop.py runs a real follower against a real leader: real gates, real clamps, real correlation. Only the network is swapped for a loopback.

Safety

SafetyConfig(
    slew=0.75,                          # max change per joint per TICK:
                                        # at the default 200 Hz that is
                                        # 150 units/s, a full sweep in ~1.3 s
    watchdog_timeout_s=0.5,             # silence for this long -> failsafe
    max_staleness_ms=100.0,             # older commands are dropped
    require_deadman=True,
    on_starvation=FailsafeAction.HOLD,
    resume_requires_reclaim=False)

follower.estop()          # unauthenticated by design: anyone can stop an arm
follower.clear_estop()    # latched: only a human on the follower host resumes

Every command is checked before it reaches the motors: who sent it, whether it arrived too late, and how far it moves the arm. If the operator stops sending, the arm holds where it is.

Recording

follower.start_recording("./sessions")
follower.start_episode("pick up the cube")
...                                          # your loop runs
follower.end_episode(success=True)
follower.stop_recording()

On the follower this captures pixels before an encoder or a network touched them. Episodes open and close on the deadman by default.

VideoSDK can also record the session in the cloud:

follower = MyArm("abcd-efgh-ijkl", cloud_recording=True)

follower.cloud_recording_state    # idle / requested / starting / ready / failed

If the cloud recorder never comes up, recording carries on locally and the state says so.

Documentation

Full documentation at docs.videosdk.live.

Apache-2.0 · docs.videosdk.live · examples