face-rhythm 0.3.1

A Python package for extracting and decomposing rhythmic facial movements from video.

Welcome to face-rhythm

• Documentation: https://face-rhythm.readthedocs.io
• Preprint: Hakim et al. (2025), bioRxiv
• Issues / support: GitHub Issues

Rhythmic facial movements from video

A Python package that turns videos of facial or other behavior into a small set of interpretable behavioral components.

Why use face-rhythm?

• Unsupervised. No labels, no model zoo.
• Interpretable. Each component is a (space × frequency × time) factor you can plot and read off directly.

How to use face-rhythm

Interactive notebooks:

• demo_pipeline.ipynb — end-to-end demo on a single session. Start here.
• demo_set_rois_multisession.ipynb — draw and align ROIs across multiple sessions of the same subject.
• demo_event_alignment.ipynb — align extracted factors to event timestamps and view trial-averaged traces.

Command line for batch runs across many sessions:

python scripts/run_pipeline_basic.py --path_params params.json --directory_save /path/to/project/

scripts/params_pipeline_basic.json is a ready-to-edit template.

Python API: see Quick start below, or the full API reference.

Installation

Requires conda or mamba. Run these commands in Terminal (Linux/macOS) or Anaconda Prompt (Windows).

0. Requirements

• Anaconda or Miniconda.
• Windows users should use the decord video backend.

1. Create a conda environment

conda create -n face_rhythm python=3.12
conda activate face_rhythm
python -m pip install --upgrade pip

Activate the env (conda activate face_rhythm) each time you use face-rhythm.

2. Install video packages

Linux:

conda install -c conda-forge 'torchcodec=*=cpu*' ffmpeg libstdcxx-ng

macOS:

conda install -c conda-forge 'torchcodec=*=cpu*' ffmpeg

Windows: skip this step.

3. Install face-rhythm

pip install face-rhythm

For headless servers, GPU acceleration, and installation troubleshooting, see the installation docs.

4. Clone the repo to get the notebooks

git clone https://github.com/RichieHakim/face-rhythm.git

CLI Quick start

import json
import face_rhythm as fr

with open("params_pipeline_basic.json", "r") as f:
    params = json.load(f)

params["project"]["directory_project"] = "/path/to/new/project/"
params["paths_videos"]["directory_videos"] = "/path/to/videos/"
params["ROIs"]["initialize"]["path_file"] = "/path/to/ROIs.h5"

results = fr.pipelines.pipeline_basic(params)

Copy scripts/params_pipeline_basic.json as a template, edit the three paths, and run. Results land in the project directory as HDF5 files plus summary plots.

Upgrading

pip install --upgrade face-rhythm

To update the cloned notebooks/scripts: cd face-rhythm && git pull.

Pipeline at a glance

1. Read the video frames (face_rhythm.helpers.BufferedVideoReader).
2. Draw ROIs that pick (a) where to track and (b) what region to crop (face_rhythm.rois).
3. Track a dense grid of points via optical flow (face_rhythm.point_tracking).
4. Compute a spectrogram for each point's trajectory (face_rhythm.spectral_analysis).
5. Factorize the (points × frequency × time) tensor with non-negative TCA (face_rhythm.decomposition).

GPU acceleration (optional)

face-rhythm runs on CPU by default. Install the CPU setup above first.

PyTorch compute: set project.use_GPU: true in your params. Check CUDA with:

python -c "import torch; print(torch.cuda.is_available())"

OpenCV CUDA: build OpenCV plus opencv_contrib with CUDA enabled, then make sure that build is the cv2 imported in this env. Useful links: OpenCV CUDA build options and opencv_contrib.

NVDEC video decoding: (uses experimental libraries). On Linux/NVIDIA systems, try a CUDA torchcodec package, then pass device='cuda' when constructing video readers:

conda install -c conda-forge 'torchcodec=*=cuda130*' ffmpeg libstdcxx-ng

Use cuda126*, cuda129*, or cuda130* to match your driver. Useful links: TorchCodec CUDA decoding and NVIDIA Video Codec SDK.

Citation

If you use face-rhythm in your research, please cite our preprint:

Hakim et al. (2025). Spectral envelopes of facial movements predict intention, cortical representations, and neural prosthetic control. bioRxiv. https://doi.org/10.1101/2025.09.10.675423

BibTeX and a machine-readable CITATION.cff are at the root of the repo.

Contributing

Bug reports, feature requests, and pull requests are welcome. Please open an issue before submitting substantial changes.

License

MIT — see LICENSE.