handex 0.1.1

Hand tracking system for robot teleoperation using MediaPipe

Hand Tracking Teleop

Real-time hand tracking for robot teleoperation using MediaPipe.

Features

• 21 hand landmarks per hand at 30fps
• Calibrated metric-space normalization
• Multiple visualization modes (2D, 3D, Rerun)
• Finger articulation tracking with curl detection
• Temporal smoothing for stable tracking

Installation

pip install handex

With visualization support:

pip install handex[vis]

Quick Start

CLI

handex                # 2D view (default)
handex -m 3d          # 3D visualization
handex -m rerun       # Rerun viewer
handex-calibrate      # Run calibration

Python API

from hand_tracking import HandDetector
import cv2

detector = HandDetector()
cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()
    if not ret:
        break

    hands = detector.detect_hands(frame)
    for hand in hands:
        # Pixel coordinates
        pixels = hand.landmarks           # (21, 3) in pixels
        # 3D metric coordinates
        meters = hand.landmarks_3d        # (21, 3) in meters
        print(f"{hand.handedness}: wrist at {meters[0]}")

Controls

• q: Quit (2D view)
• Mouse drag: Rotate 3D view
• Scroll: Zoom
• Right-click drag: Pan

License

Apache 2.0