rataGUI 0.2.0

Customizable and intuitive video acquisition system for real-time animal tracking and behavior quantification

RataGUI

RataGUI is a customizable and user-friendly Python framework for real-time animal tracking and behavioral control. At a high level, it connects video streams from cameras to online processing pipelines that can trigger external devices in real-time, enabling experiments with low latency, closed-loop feedback.

Features

RataGUI was developed with four primary design principles:

• Accessibility - RataGUI features an adaptive and intuitive graphical user interface (GUI) that makes it easier to conduct complex experiments involving heterogenous hardware without any additional programming. Using minimal dependencies and efficient multi-threaded processing, RataGUI is designed to run on a wide variety of different systems.

• Modularity - RataGUI's plug-and-play architecture enables easy customization of data processing pipelines to support your experiment's specific needs. Cameras, plugins and triggers are separated into individual modules (see tables below) and configured together in a single unified interface.

• Extensitibility - RataGUI's modular framework allows for seamless integration of user-created modules for additional functionality. You are encouraged to fork the repository and write additional modules using the provided instructions and template code. RataGUI aims to provide a platform for researchers to share code and contribute to the growing list of modules below.

• Reproducibility - RataGUI automatically logs all relevant experimental info and saves its state in a restorable, human-readable JSON format. This allows RataGUI to replicate an experiment's parameters from a single session directory.

Available Camera Modules

Name	Description	Contributor
BaslerCamera	Basler camera integration using pypylon (see installation guide)	BrainHu42
FLIRCamera	Spinnaker camera integration using PySpin (see installation guide)	BrainHu42
VideoReader	Reads video files for offline processing	BrainHu42
WebCamera	OpenCV-enabled camera integration for web or network cameras	BrainHu42

Available Plugin Modules

Name	Description	Contributor
DLCInference	Estimates animal poses using exported DeepLabCut model and writes keypoints as metadata	BrainHu42
FrameDisplay	Displays video stream in a separate window	BrainHu42
MetadataWriter	Overlays metadata onto frames and/or into a log file	BrainHu42
SleapInference	Estimates animal poses using exported SLEAP model and writes keypoints as metadata	BrainHu42
VideoWriter	Writes frames to video file using FFMPEG	BrainHu42

Available Trigger Modules

Name	Description	Contributor
NIDAQmxCounter	Writes tasks to National Instrument counter cards through the NI-DAQmx driver	BrainHu42
UDPSocket	Publishes information to a network socket using UDP protocol	BrainHu42

Installation Guide

Quick Start

conda create -n rataGUI ffmpeg pip python=3.10 cudnn=8.2 cudatoolkit=11.3 nvidia::cuda-nvcc=11.3
conda activate rataGUI
python -m pip install rataGUI

Customization Guide

RataGUI's modular framework was built for user customizability and integration. We encourage you to fork the package repository from Github and add additional modules for your specific use case.

git clone https://github.com/BrainHu42/rataGUI.git

Implement Custom Camera Models

To add another camera model, simply rename and edit the required functions provided in cameras/TemplateCamera.py to fit your camera model's specifications. RataGUI will use these functions to find, initialize, read frames from and close the camera. Any configurable camera settings should be specified in the dictionary named DEFAULT_PROPS. RataGUI will use these default settings to automatically create a dynamic menu and add it to the user interface. The configured settings will be passed into the initializeCamera function.

Implement Custom Plugins

To incorporate additional functionality, simply rename and edit the required functions provided in plugins/template_plugin.py with the custom processing needed for your use case. RataGUI will use these functions to attach your plugin to an active camera widget's processing pipeline. Any configurable plugin settings should be specified in the dictionary named DEFAULT_CONFIG. RataGUI will use these default settings to automatically create a dynamic menu and add it to the user interface.

Implement Custom Triggers

To interface with other external devices, simply rename and edit the required functions provided in triggers/template_trigger.py to fit your custom use case. RataGUI will use these functions to populate the trigger tab in the user interface with all available devices and their configurable settings. Trigger devices can be controlled through the interface as well as within a plugin process.

Contributing

If you think your module would be useful to other people, please consider submitting a merge request so that we can review and integrate your code into the main branch. We'll also add you to the list of module contributors!