Annolid

Annotate, segment, track, and analyze animals in research videos.

Annolid brings image and video annotation, instance segmentation, multi-animal tracking, keypoint tracking, and behavior analysis into a desktop application with companion command-line tools. Start with a representative labeled frame, track or propagate annotations, review difficult moments, and export results for analysis.

User guide · Getting started · Tutorials · Report an issue · YouTube

See it in action: Tracking four interacting mice with one labeled frame — a 10-minute experiment.

Table of Contents

• What You Can Do
• Quick Start
• Your First Tracking Session
• Installation Options
• Command-Line Workflows
• Annotation and Data
• Tutorials and Specialized Workflows
• Troubleshooting and Support
• Citing Annolid
• Publications
• Additional Resources
• Acknowledgements
• Contributing
• License

What You Can Do

Model backends and integrations have different dependency and hardware requirements. Follow the linked workflow guides before installing optional features.

Quick Start

The one-line installer sets up a source checkout and an isolated environment. Its default gui profile includes the GUI, machine-learning runtime, and common tracking dependencies (ml,tracking,cutie).

macOS / Linux

curl -sSL https://raw.githubusercontent.com/healthonrails/annolid/main/install.sh | bash

Windows PowerShell

irm https://raw.githubusercontent.com/healthonrails/annolid/main/install.ps1 | iex

Launch and update

After a successful default venv installation, open the install folder printed by the installer:

These shortcuts activate the environment for you. Linux may require marking the .desktop file as trusted; the adjacent .sh launcher also works from a terminal. For a manual launch, use the environment activation command printed by the installer, then run annolid.

Before updating, save your annotations and close Annolid. Update shortcuts reuse your install choices and follow the checkout's source branch. See launch and update details for update behavior and recovery guidance.

Your First Tracking Session

1. Open a short video in Annolid. You can annotate video directly; extracting frames first is optional.
2. Choose a clear seed frame and draw a polygon around each animal. Use stable instance names such as mouse_1 and mouse_2.
3. Save the annotations, then run tracking or propagation from that frame using your selected model workflow.
4. Review a short segment, especially overlap, occlusion, and fast movement. Correct masks or identities where errors begin before processing a long recording.
5. Save and export the annotations or analysis outputs you need. Add behavior events or zones when they are part of your experiment.

The Getting Started guide walks through a first session. See Core Workflows for tracking, correction, and export paths.

Installation Options

Annolid requires Python 3.10 or newer; the default GUI/core workflow supports Python 3.10–3.14. Python 3.11 or 3.12 is a practical starting point for shared lab environments. Optional model backends may have additional requirements.

Choose an installer profile

For example, on macOS or Linux:

curl -sSL https://raw.githubusercontent.com/healthonrails/annolid/main/install.sh | bash -s -- --profile workstation

See One-Line Installer Choices for Windows profile selection, CPU/GPU options, custom paths, and non-interactive installation.

Install into an existing Python environment

Activate your environment first, then install the GUI:

python -m pip install "annolid[gui]"
annolid

The package extra [gui] installs the Qt binding; it is smaller than the one-line installer's default gui profile. To include the same model extras explicitly:

python -m pip install "annolid[gui,ml,tracking,cutie]"

The installer also handles ONNX Runtime provider selection. For manual GPU setup, Conda, and the full extras list, follow the Installation guide.

Develop from source

With Git and uv available:

git clone --recurse-submodules https://github.com/healthonrails/annolid.git
cd annolid
uv venv .venv --python 3.11
uv pip install --python .venv -e ".[gui]"

Activate the environment before launching Annolid or running checks:

# macOS / Linux
source .venv/bin/activate
annolid

# Windows PowerShell
.\.venv\Scripts\Activate.ps1
annolid

Add extras needed for your work, such as .[gui,ml,tracking,cutie]. See uv Setup and Contributing for more details.

Command-Line Workflows

Run these commands in the environment where Annolid is installed:

annolid --help
annolid-run --help
annolid-run list-models
annolid-run help train
annolid-run help predict

For model-specific options, use annolid-run help train <model> or annolid-run help predict <model>, replacing <model> with a plugin name from list-models. Supported training plugins can use the YAML run-config templates; explicit CLI flags override those settings.

For example, summarize exported behavior events in 60-second bins using your project schema:

python -m annolid.behavior.time_budget exported_events.csv \
    --schema project.annolid.json \
    --bin-size 60 \
    -o time_budget.csv

See CLI Model Workflows and Annolid Agent and annolid-run for model execution and automation.

Annotation and Data

• Keep names consistent. Use stable animal identities and body-part names such as nose and tail_base. Define behavior labels before scoring and use the same definitions across annotators.
• Review tracking separately from training. A seed frame can initialize propagation; training a model across animals or recordings requires representative examples. Choose annotation volume based on reviewed results rather than a fixed frame count.
• Keep video-frame JSON and PNG files together. By default, video annotations store the image in a sidecar PNG instead of embedding it in every JSON. Set store_video_frame_data: true when self-contained frame JSON is required.
• Customize labels and flags. Use annolid --labels /path/to/labels.txt or annolid --flags "digging,rearing,grooming". Save changes with the GUI's Save action.
• Adjust label colors for review. Right-click a label in Labels or a shape in Label Instances, then select Change color. The preference is stored in app settings without changing LabelMe JSON files.
• Configure defaults. Annolid uses ~/.labelmerc (C:\Users\<username>\.labelmerc on Windows). Supply a different configuration with annolid --config /path/to/config.yaml.

See the annotation guide, behavior labeling tutorial, and zone analysis guide for detailed controls and output formats.

Tutorials and Specialized Workflows

Browse the tutorial index for notebooks and additional examples, or watch the Annolid video playlist.

Troubleshooting and Support

For help, join the Annolid Google Group. For bugs or feature requests, open an issue with your OS, Python and Annolid versions, installation method, reproduction steps, and the exact error or traceback.

Citing Annolid

If you use Annolid in your research, please cite:

• Preprint: Annolid: Annotation, Instance Segmentation, and Tracking Toolkit
• Zenodo: Find the latest release DOI via the badge at the top of this README.

@misc{yang2024annolid,
      title={Annolid: Annotate, Segment, and Track Anything You Need},
      author={Chen Yang and Thomas A. Cleland},
      year={2024},
      eprint={2403.18690},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

@article{yang2023automated,
  title={Automated Behavioral Analysis Using Instance Segmentation},
  author={Yang, Chen and Forest, Jeremy and Einhorn, Matthew and Cleland, Thomas A},
  journal={arXiv preprint arXiv:2312.07723},
  year={2023}
}

@misc{yang2020annolid,
  author = {Chen Yang and Jeremy Forest and Matthew Einhorn and Thomas Cleland},
  title = {Annolid: an instance segmentation-based multiple animal tracking and behavior analysis package},
  howpublished = {\url{https://github.com/healthonrails/annolid}},
  year = {2020}
}

Publications

• 2022 – Ultrasonic vocalization study. Pranic et al. relate mouse pup vocalizations to non-vocal behaviors (bioRxiv).
• 2022 – Digging and pain behavior. Pattison et al. link digging behaviors to wellbeing in mice (Pain, 2022).
• SfN Posters:
  • 2021: Annolid — instance segmentation-based multiple-animal tracking
  • 2023: PSTR512.01 Scoring rodent digging behavior with Annolid
  • 2023: PSTR512.02 Annolid: Annotate, Segment, and Track Anything You Need
• For more applications and datasets, visit https://cplab.science/annolid.

Additional Resources

• Example dataset (COCO format): Download from Google Drive.
• Pretrained models: Available in the shared Google Drive folder.
• Feature requests & bug reports: Open an issue at github.com/healthonrails/annolid/issues.
• Additional videos: Visit the Annolid YouTube channel for demonstrations and talks.

Acknowledgements

Annolid's tracking module integrates Cutie for enhanced video object segmentation. If you use this feature, please cite Putting the Object Back into Video Object Segmentation (Cheng et al., 2023) and the Cutie repository.

The counting tool integrates CountGD; cite the original CountGD publication and repository when you rely on this module in your research.

Contributing

Contributions are welcome! Review the contribution guidelines and engineering standard, open an issue to discuss major changes, and run relevant tests before submitting a pull request.

License

Annolid is distributed under the Creative Commons Attribution-NonCommercial 4.0 International License.