Welcome to the moseq2-app repository. Motion Sequencing (MoSeq) is an unsupervised machine learning method used to describe mouse behavior and moseq2-app is the starting point to MoSeq2 package suite.

To get started, install it with pip install moseq2-app and head over to the wiki for the full procedure and documentation for the MoSeq2 package suite.

Quick Links:

Overview

MoSeq takes 3D depth videos as input (obtained using commercially-available sensors) and then uses statistical learning techniques to identify the components of mouse body language. This is achieved by fitting an autoregressive hidden Markov model that parses behavior into a set of sub-second motifs called syllables. This segmentation naturally yields boundaries between syllables, and therefore also reveals the structure that governs the interconnections between syllables over time, which we refer to as behavioral grammar (see Wiltschko et al., 2015 for the first description of MoSeq).

Because MoSeq relies on unsupervised machine learning, MoSeq discovers the set of syllables and grammar expressed in any given experiment. By combining MoSeq with electrophysiology, multi-color photometry, and miniscope methods, neural correlates for 3D behavioral syllables have recently been identified in the dorsolateral striatum (DLS) (Markowitz et al., 2018). Furthermore, MoSeq has been combined with optogenetic stimulation to reveal the differential consequences of activating the motor cortex, the dorsal striatum, and the ventral striatum (Pisanello et al.,2017; Wiltschko et al., 2015). These results are consistent with similar results recently obtained using marker-based approaches to explore the relationship between 3D posture and activity in the posterior parietal cortex (Mimica et al., 2018).

There are two basic steps to MoSeq, and this GitHub repository and the wiki supports both. First, 3D data need to be acquired. Here you will find instructions for assembling a standard MoSeq data acquisition platform, and code to acquire 3D videos of mice as they freely behave. Second, these 3D data need to be modeled. We provide several different methods for modeling the data using MoSeq, which are compared below. We continue development of MoSeq and plan to incorporate additional features in the near future.

MoSeq2 package suite

The MoSeq2 toolkit enables users to model mouse behavior across different experimental groups, and measure the differences between their behavior usages, durations, transition patterns. etc.

This package contains functionalities that can be used interactively in jupyter notebooks. We provide a series of Jupyter Notebooks that cover the entire MoSeq pipeline to process their depth videos of mice, and segment their behavior into what is denoted as "syllables". In addition to the Jupyter notebooks, MoSeq has a Command Line Interface. Consult the wiki page for more detailed documentation of the MoSeq pipeline here.

Getting Started

pip install moseq2-app

Python 3.10 or newer. Verified on Linux; macOS and Windows support is being verified. Installing moseq2-app brings the rest of the suite with it: moseq2-extract, moseq2-pca, moseq2-model and moseq2-viz, along with their command line interfaces.

ffmpeg is a separate requirement. moseq2-extract reads and writes depth video by calling ffmpeg and ffprobe and expects to find them on your PATH. No Python install provides them, so install them yourself with conda, apt or Homebrew.

The notebooks are not part of the package. Download the two that make up the pipeline and put them in the parent directory of your data, which is what lets the interactive video widgets load:

• MoSeq2-Extract-Modeling-Notebook.ipynb
• MoSeq2-Analysis-Visualization-Notebook.ipynb

The installation guide carries the whole procedure, including how to launch Jupyter so the notebooks' video widgets work.

You can find more information about the MoSeq2 package suite, the acquisition and analysis pipeline, documentation for Command Line Interface (CLI), tutorials, etc. in the wiki. If you want to explore MoSeq functionalities, check out the guide for downloading test data.

Community Support and Contributing

• Please join to post questions and interactive with MoSeq developers and users.
• If you encounter bugs, errors or issues, please submit a Bug report here. We encourage you to check out the troubleshooting and tips section and search your issues in the existing issues first.
• If you want to see certain features in MoSeq or you have new ideas, please submit a Feature request here.
• If you want to contribute to our codebases, please check out our Developer Guidelines.
• Please tell us what you think by filling out this user survey.

Versions

• Changelog

Publications

• Mapping Sub-Second Structure in Mouse Behavior
• The Striatum Organizes 3D Behavior via Moment-to-Moment Action Selection
• Q&A: Understanding the composition of behavior
• Computational Neuroethology: A Call to Action
• Revealing the structure of pharmacobehavioral space through motion sequencing

License

MoSeq is freely available for academic use under a license provided by Harvard University. Please refer to the license file for details. If you are interested in using MoSeq for commercial purposes please contact Bob Datta directly at srdatta@hms.harvard.edu, who will put you in touch with the appropriate people in the Harvard Technology Transfer office.