pysio-hermes 0.1.0

A realtime physiological sensing and edge AI processing framework

Heterogeneous Edge Realtime Measurement and Execution System

A Unified Open-Source Framework for Realtime Multimodal Physiological Sensing, Edge AI, and Intervention in Closed-Loop Smart Healthcare Applications

Quickstart • Docs • GUI • Showcase • Cite • Contact

HERMES for the Greek mythology analogy of the god of communication and speed, protector of information, the gods' herald. He embodies the nature of smooth and reliable communication. His role accurately resonates with the vision of this framework: facilitate reliable and fast exchange of continuously generated multimodal physiological and external data across distributed wireless and wired multi-sensor hosts for synchronized realtime data collection, in-the-loop AI stream processing, and analysis, in intelligent med- and health-tech (wearable) applications.

HERMES offers out-of-the-box streaming integrations to a number of commercial sensor devices and systems, high resolution cameras, templates for extension with custom user devices, and a ready-made wrapper for easy PyTorch AI model insertion. It reliably and synchronously captures heterogeneous data across distributed interconnected devices on a local network in a continuous manner, and enables realtime AI processing at the edge toward personalized intelligent closed-loop interventions of the user. All continuously acquired data is periodically flushed to disk for as long as the system has disk space, as MKV/MP4 and HDF5 files, for video and sensor data, respectively.

Quickstart

Core

Create a Python 3 virtual environment python -m venv .venv (python >= 3.7).

Activate it with .venv/bin/activate for Linux or .venv\Scripts\activate for Windows.

Single-command install HERMES into your project along other dependendices.

pip install pysio-hermes

Extra

All the integrated, validated and supported sensor devices are separately installable as pysio-hermes-<subpackage_name>, like:

pip install pysio-hermes-torch

Will install the AI processing subpackage to wrap user-specified PyTorch models.

List of supported devices (continuously updated)

Some subpackages require OEM software installation, check each below for detailed prerequisites.

• torch Wrapper for PyTorch AI models
• pupillabs Pupil Labs Core smartglasses
• basler Basler cameras
• dots Movella DOTs IMUs
• mvn Xsens MVN Analyze MoCap suit
• awinda Xsens Awinda IMUs
• cometa Cometa WavePlus sEMG
• moticon Moticon OpenGo pressure insoles
• tmsi TMSi SAGA physiological signals
• vicon Vicon Nexus capture system
• moxy Moxy muscle oxygenation monitor

The following subpackages are in development.

• gui Live monitoring dashboard
• mic PC-attached generic microphone

Documentation

Check out the full documentation site for more usage examples, architecture overview, detailed extension guide, and FAQs.

Data Annotation

We developed PysioViz a complementary dashboard based on Dash Plotly for analysis and annotation of the collected multimodal data. We use it ourselves to generate ground truth labels for the AI training workflows. Check it out and leave feedback!

Showcase

These are some of our own projects enabled by HERMES to excite you to adopt it in your smart closed-looop healthtech usecases.

AI-enabled intent prediction for high-level locomotion mode selection in a smart leg prosthesis

Realtime automated cueing for freezing-of-gait Parkinson's patients in free-living conditions

Personalized level of assistance in prolong use rehabilitation and support exoskeletons

License

This project is licensed under the MIT license - see the LICENSE file for details.

Citation

When using in your project, research, or product, please cite the following and notify us so we can update the index of success stories enabled by HERMES.

Acknowledgement

This project was primarily written by Maxim Yudayev while at the Department of Electrical Engineering, KU Leuven.

This study was funded, in part, by the AidWear project funded by the Federal Public Service for Policy and Support, the AID-FOG project by the Michael J. Fox Foundation for Parkinson’s Research under Grant No.: MJFF-024628, and the Flemish Government under the Flanders AI Research Program (FAIR).

HERMES is a "Ship of Theseus"[1] of ActionSense that started as a fork and became a complete architectural rewrite of the system from the ground up to bridge the fundamental gaps in the state-of-the-art, and to match our research group's needs in realtime deployments and reliable data acquisition. Although there is no part of ActionSense in HERMES, we believe that its authors deserve recognition as inspiration for our system.

Special thanks for contributions, usage, bug reports, good times, and feature requests to Juha Carlon (KU Leuven), Stefano Nuzzo (VUB), Diwas Lamsal (KU Leuven), Vayalet Stefanova (KU Leuven), Léonore Foguenne (ULiège).