Welcome to face-rhythm

• Documentation: https://face-rhythm.readthedocs.io
• Preprint: Hakim et al. (2025), bioRxiv
• Issues / support: GitHub Issues

What is face-rhythm

A Python package that turns videos of facial or other behavior into a small set of interpretable behavioral components.

Why use face-rhythm?

• Unsupervised. No labels, no model zoo.
• Interpretable. Each component is a (space × frequency × time) factor you can plot and read off directly.

How to use it

Interactive notebooks:

• demo_pipeline.ipynb — end-to-end demo on a single session. Start here.
• demo_set_rois_multisession.ipynb — draw and align ROIs across multiple sessions of the same subject.
• demo_event_alignment.ipynb — align extracted factors to event timestamps and view trial-averaged traces.

Command line for batch runs across many sessions:

python scripts/run_pipeline_basic.py --path_params params.json --directory_save /path/to/project/

scripts/params_pipeline_basic.json is a ready-to-edit template.

Python API: see Quick start below, or the full API reference.

Installation

0. Requirements

• Anaconda or Miniconda or Mamba

1. Create a conda environment

conda create -n face_rhythm python=3.12
conda activate face_rhythm
python -m pip install --upgrade pip

Activate the env (conda activate face_rhythm) each time you use face-rhythm.

2. Install video packages

Linux:

conda install -c conda-forge 'torchcodec=*=cpu*' ffmpeg libstdcxx-ng

macOS:

conda install -c conda-forge 'torchcodec=*=cpu*' ffmpeg

Windows: skip this step. torchcodec doesn't explicitly support Windows. Installing it often works, but is not guaranteed. Unless you need ultrafast GPU speeds, just use the 'decord' backend, instead.

3. Install face-rhythm

pip install face-rhythm

For headless servers, GPU acceleration, and installation troubleshooting, see the installation docs.

4. Clone the repo to get the notebooks

git clone https://github.com/RichieHakim/face-rhythm.git

CLI Quick start

import json
import face_rhythm as fr

with open("params_pipeline_basic.json", "r") as f:
    params = json.load(f)

params["project"]["directory_project"] = "/path/to/new/project/"
params["paths_videos"]["directory_videos"] = "/path/to/videos/"
params["ROIs"]["initialize"]["path_file"] = "/path/to/ROIs.h5"

results = fr.pipelines.pipeline_basic(params)

Copy scripts/params_pipeline_basic.json as a template, edit the three paths, and run. Results land in the project directory as HDF5 files plus summary plots.

Upgrading

pip install --upgrade face-rhythm

To update the cloned notebooks/scripts: cd face-rhythm && git pull.

Pipeline at a glance

1. Read the video frames (face_rhythm.helpers.BufferedVideoReader).
2. Draw ROIs that pick (a) where to track and (b) what region to crop (face_rhythm.rois).
3. Track a dense grid of points via optical flow (face_rhythm.point_tracking).
4. Compute a spectrogram for each point's trajectory (face_rhythm.spectral_analysis).
5. Factorize the (points × frequency × time) tensor with non-negative TCA (face_rhythm.decomposition).

GPU acceleration (optional)

face-rhythm runs on CPU by default. Install the CPU setup above first.

PyTorch compute: set project.use_GPU: true in your params. Check CUDA with:

python -c "import torch; print(torch.cuda.is_available())"

OpenCV CUDA: build OpenCV plus opencv_contrib with CUDA enabled, then make sure that build is the cv2 imported in this env. Useful links: OpenCV CUDA build options and opencv_contrib.

NVDEC video decoding: (uses experimental libraries). On Linux/NVIDIA systems, try a CUDA torchcodec package, then pass device='cuda' when constructing video readers:

conda install -c conda-forge 'torchcodec=*=cuda130*' ffmpeg libstdcxx-ng

Use cuda126*, cuda129*, or cuda130* to match your driver. Useful links: TorchCodec CUDA decoding and NVIDIA Video Codec SDK.

Citation

If you use face-rhythm in your research, please cite our preprint:

Hakim et al. (2025). Spectral envelopes of facial movements predict intention, cortical representations, and neural prosthetic control. bioRxiv. https://doi.org/10.1101/2025.09.10.675423

BibTeX and a machine-readable CITATION.cff are at the root of the repo.

Contributing

Bug reports, feature requests, and pull requests are welcome. Please open an issue before submitting substantial changes.

License

MIT — see LICENSE.