GitHub  •  LinkedIn  •  X  •  Discord

medulla: Hardware Communication Skills

medulla is a module in the Telekinesis SDK for connecting to cameras and hardware devices. It provides tools to interface with various 2D and 3D cameras and integrate them into Telekinesis applications. It includes:

• 2D and 3D camera interfacing
• Data acquisition and preprocessing
• Time-synchronized sensor streams
• Integration with Telekinesis modules (vitreous, retina, cornea, pupil)

This library is used for robotics applications that require camera connectivity, including vision pipelines, multi-camera robot perception, and Physical AI agent integration.

medulla is currently in its early development phase (pre-1.0). There will be continuous minor version updates that may introduce new features and improvements. To ensure compatibility and have the latest features, please always install or upgrade to the latest version of the package.

Installation

1. Create an isolated environment to avoid dependency conflicts. We recommend installing Miniconda by following the instructions from here.

2. Create a new conda environment called telekinesis-medulla:

   conda create -n telekinesis-medulla python=3.11
   

3. Activate the environment:

   conda activate telekinesis-medulla
   

4. Install medulla using pip:

   We currently support Python versions - 3.11, 3.12. Ensure your environment is in the specified Python version.

   pip install telekinesis-medulla
   

5. Install the additional dependencies to run the official examples:

   pip install telekinesis-medulla[examples]
   

Additional Setup

Camera drivers and vendor SDKs require additional installation steps. Please follow the official documentation for vendor-specific SDK setup.

IDS cameras: Installing the ids extras (pip install telekinesis-medulla[ids]) fetches ids_peak, ids_peak_ipl, and ids_peak_icv directly from IDS Imaging Development Systems GmbH's own PyPI distribution. By installing these packages, you become the licensee under the IDS Software Suite License Terms and are bound by its conditions.

RealSense cameras: Installing the realsense extras (pip install telekinesis-medulla[realsense]) fetches pyrealsense2 directly from Intel's PyPI distribution. By installing this package, you become the licensee under the Intel RealSense SDK 2.0 License Terms and are bound by its conditions.

Device factories

Runnable construction examples are in examples/factory: sensor, conveyors, and CNC machine. They leave devices unconnected and do not command motion.

Create devices from shared, validated datatype configurations:

from telekinesis.datatypes import (
    CNCMachineConfig, ConveyorConfig, LightbeamSensorConfig,
)
from telekinesis.medulla.factory import create_conveyors, create_machine, create_sensor

belts = create_conveyors(ConveyorConfig(name="infeed", class_name="Conveyor"))
sensor = create_sensor(LightbeamSensorConfig(name="entry", class_name="LightBeamSensor"))
machine = create_machine(CNCMachineConfig(name="cnc", class_name="CNCMachine"))

create_conveyors returns one belt per prim_paths entry, in the same order, named <name>__<zero-based index>. The other factories return one device with the configured name. All three validate before loading the selected driver.

Devices are returned unconnected: apply connection and operating settings in your application. These calls do not stream, load scenes, or actuate devices; sensor construction does create BabyROS endpoints. Callers own cleanup. The initial implementations support Isaac Sim connections.

Example

Run a sample Python script to quickly test your installation.

1. Create a Python file named medulla_example.py in a directory of your choice and paste the following:

   from telekinesis.medulla.cameras import webcam
   
   camera = webcam.Webcam(name="my_webcam_camera", camera_id=0)
   camera.connect()
   image = camera.capture_color_image()
   print(image)
   camera.disconnect()
   

2. Activate the previously created conda environment. On a terminal, navigate to the directory where medulla_example.py was created and run:

   python medulla_example.py
   

   Expected output: Some logs and an image from the connected webcam in the below format:

    ...
    [[[158 155 140]
    [159 155 142]
    [160 154 144]
    ...
    [255 255 252]
    [255 255 251]
    [255 255 251]]
    ...
   

Supported Cameras

Resources

• Examples Runnable usage examples for Medulla: Telekinesis Examples

• Documentation Full SDK documentation and usage details: Telekinesis Documentation

• Sample Data Datasets used across the examples: Telekinesis Data

Support

For issues and questions:

• Create an issue in the GitHub repository.
• Contact the Telekinesis development team at support@telekinesis.ai or on Discord.