GitHub  •  LinkedIn  •  X  •  Discord

Telekinesis Agentic Skill Library

The Telekinesis Agentic Skill Library is the first large-scale Python library for building agentic robotics, computer vision, and Physical AI systems. It provides:

• Skills: a broad set of AI algorithms for perception, motion planning, and control.
• Physical AI Agents: LLM/VLM agents for task planning across industrial, mobile, and humanoid robots.

The library is intended for robotics, computer vision, and research teams that want to:

• Speed up development by integrating production-grade robotics, computer vision, and AI algorithms
• Add intelligence to robots with LLM/VLM-driven task planning tied to real perception and control systems
• Iterate quickly on Physical AI systems using a single, consistent Python library.

Learn more about the Telekinesis Agentic Skill Library in the About Telekinesis.

Join our Discord community to add your own skills and be part of the Physical AI revolution!

Release Model

The Telekinesis Agentic Skill Library is currently in active development (pre-1.0).
Modules are introduced incrementally, and the API may evolve between minor releases. To ensure compatibility and access to the latest capabilities, always install or upgrade to the most recent version of the package.

Currently available modules:

• cornea
• retina
• pupil
• vitreous
• medulla
• synapse
• babyros
• datatypes
• isaacsim

Installation

Core SDK

1. Create an isolated environment so that there is no dependency conflicts. We recommend installing Miniconda environment by following instructions from here.

2. Create a new conda environment called telekinesis:

   conda create -n telekinesis python=3.11
   

3. Activate the environment:

   conda activate telekinesis
   

4. Install the core SDK using pip:

   We currently support Python versions - 3.11, 3.12. Ensure your environment is in the specified Python version.

   pip install telekinesis-ai
   

   Note: The Python module is called telekinesis, while the package published on PyPI is telekinesis-ai.

medulla

medulla is a module in the Telekinesis SDK for connecting to cameras and hardware devices. It is installed by default as part of telekinesis-ai, with webcam support included out of the box:

pip install telekinesis-ai

In order to install vendor specific dependencies, please follow the official documentation on https://docs.telekinesis.ai/medulla/overview.html, e.g. for using IDS cameras with medulla:

pip install telekinesis-ai[medulla-ids]

You can find the list of supported vendors in the overview section under supported cameras.

See the official medulla documentation for more details about the installation.

IDS cameras: Installing the ids extras (pip install telekinesis-ai[medulla-ids]) fetches ids_peak, ids_peak_ipl, and ids_peak_icv directly from IDS Imaging Development Systems GmbH's own PyPI distribution. By installing these packages, you become the licensee under the IDS Software Suite License Terms and are bound by its conditions.

RealSense cameras: Installing the realsense extras (pip install telekinesis-ai[medulla-realsense]) fetches pyrealsense2 directly from Intel's PyPI distribution. By installing this package, you become the licensee under the Intel RealSense SDK 2.0 License Terms and are bound by its conditions.

synapse

synapse is a module in the Telekinesis SDK for controlling and communicating with industrial robot manipulators and tools. It provides a unified API across multiple robot and tool brands. It is installed by default as part of telekinesis-ai.

Robot support:

• Offline (simulation): all supported robots
• Real-world (hardware control): Universal Robots only

Supported platforms: Windows, macOS

synapse depends on telekinesis-urdfs for robot model data, which must be installed before telekinesis-ai.

1. Install telekinesis-urdfs:

   git clone --depth 1 https://github.com/telekinesis-ai/telekinesis-urdfs.git
   cd telekinesis-urdfs
   pip install .
   

   Note: telekinesis-urdfs is a large repository containing robot model data. The initial clone and wheel build are expected to take several minutes — do not interrupt the process.

2. Install telekinesis-ai (which includes synapse):

   pip install telekinesis-ai
   

See the official synapse documentation for more details about the installation and supported robots.

isaacsim

isaacsim is currently under active development. It is a module in the Telekinesis SDK for connecting to NVIDIA Isaac Sim through the Telekinesis Isaac Sim bridge (a Python client built on a C++ core). It is an optional dependency — install it with the isaacsim extra:

pip install telekinesis-ai[isaacsim]

Supported platforms: Windows (macOS and Linux support coming soon)

Example usage:

from telekinesis.isaacsim_client import IsaacSimClient

client = IsaacSimClient(
    api_key="",
    base_url="http://127.0.0.1:8766",
    websocket_base_url="ws://127.0.0.1:8766",
)

created = client.articulation.create("/World/ur10e")
print(created["articulation_id"])

client.articulation.set_joint_positions(created["articulation_id"], [0.0] * created["num_dof"])

More examples are available on Telekinesis Isaac Sim Examples, which is also under active development.

Getting Started

Telekinesis SDK requires a free API key to authenticate requests.

Create one at platform.telekinesis.ai. See the Quickstart for more details on the generation of API key.

Continue to Example section to quickly validate the installation.

Example

The following example assumes the API key has been generated and has been set as TELEKINESIS_API_KEY environment variable.

Run a sample python code to quickly test your installation.

This example will fail if TELEKINESIS_API_KEY is not set correctly.

1. Create a Python file named telekinesis_ai_example.py in a directory of your choice in your system, and copy paste the below:

   import numpy as np
   from telekinesis import vitreous
   
   # Create a cylinder mesh
   cylinder_mesh = vitreous.create_cylinder_mesh(
   		radius=0.01,
   		height=0.02,
   		radial_resolution=20,
   		height_resolution=4,
   		retain_base=False,
   		vertex_tolerance=1e-6,
   		transformation_matrix=np.eye(4, dtype=np.float32),
   		compute_vertex_normals=True,
   	)
   
   # Convert it to point cloud
   point_cloud = vitreous.convert_mesh_to_point_cloud(
   		mesh=cylinder_mesh,
   		num_points=10000,
   		sampling_method="poisson_disk",
   		initial_sampling_factor=5,
   		initial_point_cloud=None,
   		use_triangle_normal=False,
   	)
   print(point_cloud.positions)
   # Use point_cloud in downstream processing or visualize the point cloud with any tool
   

2. On a terminal, navigate to the directory where the above file named telekinesis_ai_example.py has been created, run the below command:

   python telekinesis_ai_example.py
   

   Expected output: Some logs and random valued point cloud positions in the below format is output

   ...
   ...
   [[-0.00835031 -0.00536731 -0.00429686]
    [ 0.00854885  0.00497764  0.00044501]
    [ 0.00838172  0.00530565  0.00249433]
    ...
    [-0.00280485  0.00955575  0.00949276]
    [-0.00743726 -0.00653076 -0.00238814]
    [ 0.00023231 -0.00996321  0.00887559]]
   

You are now set up to build with Telekinesis.

The recommended way to explore Telekinesis Agentic Skill Library today is via the Telekinesis Examples repository, which contains fully runnable workflows built on top of the SDK.

Resources

• Examples
  Runnable examples demonstrating Telekinesis Agentic Skill Library capabilities: Telekinesis Examples

• Documentation
  Full SDK documentation and usage details: Telekinesis Documentation

• Sample Data
  Datasets used across the examples: Telekinesis Data

Support

For issues and questions:

• Create an issue in the GitHub repository.
• Contact the Telekinesis development team at support@telekinesis.ai or on Discord.