sinapsis-cotracker 0.1.6

Templates for multi-object tracking using the CoTracker model, supporting offline, online, and visualization workflows.

Sinapsis CoTracker

Templates for multi-object tracking and visualization using the CoTracker model

🐍 Installation • 🚀 Features • 📙 Documentation • 🔍 License

The siinapsis-cotracker module provides a robust and flexible implementation for multi-object tracking using the CoTracker model. It enables users to easily configure and run tracking pipelines for video input processing and visualization tasks.

🐍 Installation

Install using your package manager of choice. We encourage the use of uv

[!WARNING] cotracker dependency is required to install sinapsis-cotracker

UV instructions

Install cotracker in your working environment as follows:

uv pip install git+https://github.com/facebookresearch/co-tracker.git

then install sinapsis-cotracker

uv pip install sinapsis-cotracker --extra-index-url https://pypi.sinapsis.tech

Raw pip instructions

Install cotracker in your working environment as follows:

pip install git+https://github.com/facebookresearch/co-tracker.git

then install sinapsis-cotracker

pip install sinapsis-cotracker --extra-index-url https://pypi.sinapsis.tech

🚀 Features

Templates Supported

The Sinapsis CoTracker module offers a set of templates for multi-object tracking and visualization using the CoTracker model. These templates enable users to perform both online and offline tracking, process video inputs, and visualize tracking results on video frames. The templates in this package include functionality for:

• CoTrackerOffline: Handles offline multi-object tracking for short videos. It loads the entire video into memory for processing and is the simplest method for offline tasks.
• CoTrackerOfflineLarge: Provides offline tracking for long videos by using a memory-efficient, incremental engine.
• CoTrackerOnline: Supports real-time object tracking with advanced grid query and support grid features.
• CoTrackerVisualizer: Visualizes tracking results with customizable trace, line width, and visualization modes.

[!TIP] Use CLI command sinapsis info --all-template-names to show a list with all the available Template names installed with Sinapsis CoTracker.

[!TIP] Use CLI command sinapsis info --example-template-config TEMPLATE_NAME to produce an example Agent config for the Template specified in TEMPLATE_NAME.

For example, for CoTrackerOffline use sinapsis info --example-template-config CoTrackerOffline to produce the following example config:

agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: CoTrackerOffline
  class_name: CoTrackerOffline
  template_input: InputTemplate
  attributes:
    device: '`replace_me:typing.Literal[''cuda'', ''cpu'']`'
    generic_key_field: CoTrackerResults
    model_cache_dir: /home/cv/.cache/sinapsis
    model_variant: '`replace_me:typing.Literal[''baseline'', ''scaled'']`'
    use_segmentation_mask: false
    grid_size: '`replace_me:<class ''int''>`'
    grid_query_frame: 0
    backward_tracking: false

📚 Example Usage

Below is an example YAML configuration for processing a video file and visualizing tracking results using Sinapsis CoTracker templates. This setup loads a video with the VideoReaderCV2, performs real-time object tracking with the CoTrackerOnline template, visualizes the results with the CoTrackerVisualizer, and saves the output as a new video file using the VideoWriterCV2.

Config

agent:
  name: cotracker_agent

templates:
  - template_name: InputTemplate
    class_name: InputTemplate
    attributes: {}

  - template_name : VideoReaderCV2
    class_name: VideoReaderCV2
    template_input: InputTemplate
    attributes:
      video_file_path : "artifacts/palace.mp4"
      batch_size: 16

  - template_name: CoTrackerOnline
    class_name: CoTrackerOnline
    template_input: VideoReaderCV2
    attributes:
      model_variant: baseline
      device: cuda
      grid_size: 15

  - template_name: CoTrackerVisualizer
    class_name: CoTrackerVisualizer
    template_input: CoTrackerOnline
    attributes:
      device : cuda
      linewidth: 3
      overwrite: true

  - template_name: VideoWriterCV2
    class_name: VideoWriterCV2
    template_input: CoTrackerVisualizer
    attributes:
      destination_path: "artifacts/result.mp4"
      height: -1
      width: -1
      fps: 30

This configuration defines an agent and a sequence of templates for video processing, object tracking, and visualization.

IMPORTANT: The VideoReaderCV2 and VideoWriterCV2 templates are part of the sinapsis-data-readers and sinapsis-data-writers packages, respectively. To use this example, ensure that you have installed these packages.

To run the config, use the CLI:

sinapsis run name_of_config.yml

📙 Documentation

Documentation for this and other sinapsis packages is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.