mosaic

A platform for behavior analysis.

Mosaic drives trackers — TRex, SLEAP, Lightning Pose, Ultralytics — or imports tracks something else already produced, then gives a flexible and expandable analysis framework for working with the data through the feature construct: individual behavioral metrics, collective motion, unsupervised methods, and supervised model fitting on annotated behaviors. A core focus is enabling multi-animal analysis: a feature can be calculated on individuals, on pairs, or on all animals at once.

Results are organized by a content-addressed scheme for run-caching and reproducibility: the same inputs and parameters produce the same run_id, so re-running is a no-op, parameter sweeps organize themselves, and mosaic inventory tells you exactly what a dataset already holds.

What it does

Install

conda create -n mosaic python=3.12 -y
conda activate mosaic
conda install -c conda-forge ffmpeg av py-opencv -y
pip install "mosaic-behavior[all]"

The distribution is mosaic-behavior and the import name is mosaic. An unrelated project named mosaic exists on PyPI, so never run pip install mosaic.

pip install mosaic-behavior on its own is a complete analysis install: converters, features, clustering, classifiers, overlays. [all] adds the deep-learning surface: the heatmap localizer and the identity models, which means PyTorch. YOLO pose and POLO point work is not in it. Tracking, inference and training alike run in an environment you build, so no install of mosaic carries an AGPL-licensed dependency.

ffmpeg and ffprobe must be on PATH. av and py-opencv are installed from conda-forge, before pip runs, so that the environment holds a single ffmpeg build. A plain pip install brings the PyPI wheels of both, which collide on macOS.

Install from source

To work on mosaic itself, or to build the Ultralytics, POLO, keypoint-MoSeq or VAME environment, clone the repository and install it editable into the same kind of conda environment:

git clone https://github.com/EcodylicScience/mosaic.git
cd mosaic
pip install -e ".[all]"

The environment notes of CONTRIBUTING.md cover working against an unreleased mosaic-media.

Installation has the extras table, a CPU-only PyTorch line, the tools mosaic drives but does not install, and Windows/WSL2 support. The Ultralytics, POLO, keypoint-MoSeq and VAME environments are built from files in the repository that the pip package does not carry.

In 60 seconds

# Work in a directory of your own, not inside a mosaic source checkout.
cd ~
mosaic init study --name "Experiment 1"

# Point the dataset at video that can live anywhere -- a NAS, another volume.
mosaic sources add -m study/dataset.yaml --kind media \
    --path /data/day1 --extensions .mp4
mosaic scan -m study/dataset.yaml

# Track it, then derive something from the result.
mosaic track trex -m study/dataset.yaml --set track_max_individuals=4
mosaic run  -m study/dataset.yaml --feature speed-angvel

# What does this dataset now hold, and is any of it stale?
mosaic inventory -m study/dataset.yaml

Already have tracks? Skip the tracker: declare them as a --kind tracks source with their --src-format, and mosaic convert-tracks standardizes them.

Documentation

Five worked examples live in notebooks/ in the repository, not in the pip package. Each fetches its own data from Hugging Face, so they run as written:

The two tracking notebooks drive external tools mosaic does not install; they say so and skip those sections when the tool is absent.

Status

Early development (0.x). Public APIs, feature names, and on-disk layouts may change between releases; CHANGELOG.md records what moved and why.

How to cite

Cite the tools that produced your results first: the tracker, the pose model and the method behind each feature you used. Cite mosaic second, as the software that ran them. The citing page says how, and NOTICE records the origin, license and citation of each component.

Contributing

See CONTRIBUTING.md, and please open an issue before making large changes. To contribute with an AI coding agent, CLAUDE.md is the developer orientation.

License

GNU Affero General Public License v3 or later. See LICENSE, and NOTICE for the third-party attributions that must travel with it.

mosaic bundles no third-party source and no model weights, but it drives tools whose terms differ from its own — keypoint-MoSeq prohibits commercial use, TRex requires a paid license for company use, and Ultralytics is AGPL-3.0. NOTICE records how each component reaches you and what that means for your use of it.