lightning-pose 2.3.0

Semi-supervised pose estimation using pytorch lightning

Lightning Pose is an end-to-end package designed for robust multi-view and single-view animal pose estimation using advanced transformer architectures. It leverages Multi-View Transformers and simulated occlusions to learn geometric relationships between views, resulting in strong performance on occlusions (Aharon, Whiteway et al., 2026). For single-view datasets it leverages temporal context and unsupervised losses for strong performance in challenging scenarios (Biderman, Whiteway et al. 2024, Nature Methods). It has a rich GUI that supports the end-to-end workflow: labeling, model management, and evaluation.

Below is a quick start guide; see the full documentation here.

Installation

Lightning Pose requires a Linux or WSL environment with an NVIDIA GPU. It has been tested on Ubuntu 18.04, 22.04, and 24.04 with Python versions 3.10-3.12.

For users without access to a local NVIDIA GPU, it is highly recommended to use the Lightning AI cloud environment, which provides a persistent, browser-based "Studio" with on-demand access to powerful GPUs and pre-configured CUDA environments.

Install dependencies:

sudo apt install ffmpeg

# Verify nvidia-driver with CUDA 12+
nvidia-smi

In a clean python virtual environment (conda or other virtual environment manager), run:

pip install lightning-pose lightning-pose-app

That's it! Installation should take 5-10 minutes depending on your internet connection. To run the app:

litpose run_app

Please see the installation guide for more detailed instructions, and feel free to reach out to us on Discord in case of any hiccups.

Getting Started

To get started with Lightning Pose, follow the guides on our documentation:

• Create your first project using the app
• Follow the CLI User Guides (Singleview, Multiview)
• Test out the API in the demo noteboook

Note that model training time will depend heavily on dataset size and GPU resources. Fitting a typical backbone (ResNet-50 or ViT-Small) on ~200 labeled frames will take about 20 minutes on a T4 GPU. See the publications linked above for more thorough benchmarking of training and inference times.

Community

The Lightning Pose team also actively develops the Ensemble Kalman Smoother (EKS), a simple and performant post-processor that works with any pose estimation package including Lightning Pose, DeepLabCut, and SLEAP.

Lightning Pose is primarily maintained by Karan Sikka (Columbia University) and Matt Whiteway (Columbia University).

Lightning Pose is under active development and we welcome community contributions. Whether you want to implement some of your own ideas or help out with our development roadmap, please get in touch with us on Discord (see contributing guidelines here).

Funding

We are grateful for support from the following:

• Gatsby Charitable Foundation GAT3708
• NIH R50NS145433
• NIH U19NS123716
• NSF 1707398
• The NSF AI Institute for Artificial and Natural Intelligence
• Simons Foundation
• Wellcome Trust 216324
• Zuckerman Institute (Columbia University) Team Science