Skip to main content


Sinapsis
Sinapsis Cotracker

Track points and objects across frames with CoTracker.

Overview

sinapsis-cotracker — Templates for multi-object tracking using the CoTracker model, supporting offline, online, and visualization workflows.

Point tracking provides trajectories through a video, even when an application does not start from frame-by-frame object detections. The package includes offline, online and visualisation workflows for different data access patterns.

Where it fits

Online tracking is useful for streams; offline methods can use full temporal context. Video resolution, sequence length and model state affect memory use.

Installation

Requires Python 3.13 or later, as declared in this package’s manifest. Install the base distribution using your preferred package manager:

uv pip install sinapsis-cotracker
# or
pip install sinapsis-cotracker

Optional features and dependencies

The base installation includes the dependencies required by the package. The following extras are declared in this package’s pyproject.toml; install only those needed by your workflow. They are standard Python optional dependencies, not separate execution profiles.

Extra Adds / when to select it
all Bundle declared by this package; its manifest includes sinapsis-data-readers, sinapsis-data-visualization, sinapsis-data-writers.
sinapsis-data-readers Load external media or datasets into DataContainers. Dependencies: sinapsis-data-readers[opencv]
sinapsis-data-writers Save the resulting DataContainer content to files. Dependencies: sinapsis-data-writers
sinapsis-data-visualization Render plots or annotate results for inspection. Dependencies: sinapsis-data-visualization[visualization-matplotlib]

For example:

uv pip install "sinapsis-cotracker[sinapsis-data-readers]"

The manifest also defines development-only dependency groups: dev (ruff, pre-commit). These support source development and testing and are not runtime extras.

🚀 Features

Templates Supported

The Sinapsis CoTracker module offers a set of templates for multi-object tracking and visualization using the CoTracker model. These templates enable users to perform both online and offline tracking, process video inputs, and visualize tracking results on video frames. The templates in this package include functionality for:

  • CoTrackerOffline: Handles offline multi-object tracking for short videos. It loads the entire video into memory for processing and is the simplest method for offline tasks.
  • CoTrackerOfflineLarge: Provides offline tracking for long videos by using a memory-efficient, incremental engine.
  • CoTrackerOnline: Supports real-time object tracking with advanced grid query and support grid features.
  • CoTrackerVisualizer: Visualizes tracking results with customizable trace, line width, and visualization modes.

For example, for CoTrackerOffline use sinx templates example-config CoTrackerOffline to produce the following example config:

agent:
  name: my_sinapsis_cotracker_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: CoTrackerOffline
  class_name: CoTrackerOffline
  template_input: InputTemplate
  attributes:
    device: '`replace_me:typing.Literal[''cuda'', ''cpu'']`'
    generic_key_field: CoTrackerResults
    model_cache_dir: /home/cv/.cache/sinapsis
    model_variant: '`replace_me:typing.Literal[''baseline'', ''scaled'']`'
    use_segmentation_mask: false
    grid_size: '`replace_me:<class ''int''>`'
    grid_query_frame: 0
    backward_tracking: false
📚 Example Usage

Below is an example YAML configuration for processing a video file and visualizing tracking results using Sinapsis CoTracker templates. This setup loads a video with the VideoReaderCV2, performs real-time object tracking with the CoTrackerOnline template, visualizes the results with the CoTrackerVisualizer, and saves the output as a new video file using the VideoWriterCV2.

Config
agent:
  name: cotracker_agent

templates:
  - template_name: InputTemplate
    class_name: InputTemplate
    attributes: {}

  - template_name : VideoReaderCV2
    class_name: VideoReaderCV2
    template_input: InputTemplate
    attributes:
      video_file_path : "artifacts/palace.mp4"
      batch_size: 16

  - template_name: CoTrackerOnline
    class_name: CoTrackerOnline
    template_input: VideoReaderCV2
    attributes:
      model_variant: baseline
      device: cuda
      grid_size: 15

  - template_name: CoTrackerVisualizer
    class_name: CoTrackerVisualizer
    template_input: CoTrackerOnline
    attributes:
      device : cuda
      linewidth: 3
      overwrite: true

  - template_name: VideoWriterCV2
    class_name: VideoWriterCV2
    template_input: CoTrackerVisualizer
    attributes:
      destination_path: "artifacts/result.mp4"
      height: -1
      width: -1
      fps: 30

This configuration defines an agent and a sequence of templates for video processing, object tracking, and visualization.

IMPORTANT: The VideoReaderCV2 and VideoWriterCV2 templates are part of the sinapsis-data-readers and sinapsis-data-writers packages, respectively. To use this example, ensure that you have installed these packages.

To run the config, use the CLI:

sinx jobs run my_agent.yaml

Example Agent

The following is adapted directly from the package’s shipped cotracker_batch.yml configuration. Save it as my_cotracker_agent.yaml. The example may require model checkpoints, input files, additional Template packages, or provider credentials as described in its configuration.

agent:
  name: my_cotracker_agent
  description: >
    "Agent to load video from local dir, perform CoTrackerOnline inference
    and store visualization results in destination path."

templates:
  - template_name: InputTemplate
    class_name: InputTemplate
    attributes: {}

  - template_name : VideoReaderCV2
    class_name: VideoReaderCV2
    template_input: InputTemplate
    attributes:
      root_dir: "."
      video_file_path : "artifacts/apple.mp4"
      batch_size: -1

  - template_name: CoTrackerOfflineLarge
    class_name: CoTrackerOfflineLarge
    template_input: VideoReaderCV2
    attributes:
      model_variant: baseline
      device: cuda
      grid_size: 15

  - template_name: CoTrackerVisualizer
    class_name: CoTrackerVisualizer
    template_input: CoTrackerOfflineLarge
    attributes:
      device : cuda
      linewidth: 3
      overwrite: true
      generic_key_field: CoTrackerOfflineLarge

  - template_name: VideoWriterCV2
    class_name: VideoWriterCV2
    template_input: CoTrackerVisualizer
    attributes:
      root_dir: "."
      destination_path: "artifacts/result.mp4"
      height: -1
      width: -1
      fps: 10
sinx jobs run my_cotracker_agent.yaml

More configurations: cotracker_batch.yml, cotracker_online.yml, cotracker_offline.yml.

Documentation

License

Sinapsis open-source components are available under your choice of the MIT License or the Apache License 2.0, unless otherwise noted.

Metadata

Release files for sinapsis-cotracker 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sinapsis-cotracker 1.0.0
File Size Uploaded Provenance Actions
sinapsis_cotracker-1.0.0.tar.gz 106.4 kB Without provenance Details for sinapsis_cotracker-1.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for sinapsis-cotracker 1.0.0
File Interpreter ABI Platform Actions
sinapsis_cotracker-1.0.0-py3-none-any.whl Python 3 none any Details for sinapsis_cotracker-1.0.0-py3-none-any.whl

Total release size: 130.0 kB

Release files / sinapsis_cotracker-1.0.0.tar.gz

Download URL sinapsis_cotracker-1.0.0.tar.gz
Size 106.4 kB
Tags Source
SHA-256 checksum
How to use checksums
3f43ae6279016688f6838ede191f05979ebcfd803aac517a04bd1b3b1a738195
BLAKE2b-256 checksum
How to use checksums
f10cb9720d5aea2d1489a61394f3f4f76db31f296dfc7f822490b56a7a58942a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.6.17

Release files / sinapsis_cotracker-1.0.0-py3-none-any.whl

Download URL sinapsis_cotracker-1.0.0-py3-none-any.whl
Size 23.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
7921a3b604081329760e70c42e88f3e2def074b52a515e468a3000c59ce92cc9
BLAKE2b-256 checksum
How to use checksums
576c20712df7a96e8f3616808e119b3d25aaaf40ad2c9a17f66476b08f3d6509
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.6.17

Security for sinapsis-cotracker 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page