SleePyPhases 0.3.1

A framwork for creating deep learning pipelines for sleep data

Arousal Detector

Machine learning based arousal detection during sleep. This project uses pyPhases.

Setup

Dataset

You need to download a valid dataset for example SHHS or MESA.

Supported dataset and their identifiers:

• SHHS: shhs (SHHS1)
• MESA: mesa
• MrOS: mros
• TSM: tsm (local dataset from Universitätsklinikum Carl Gustav Carus Dresden)

Setup dataset path

Depending on the dataset following config Values need to be specified: shhs-path, mros-path, mesa-path, tsm-path

It is recommended to set up a file userconfigs/local.yml with following configuration values:

Extract:
  useMultiThreading: True  # "True" default value
  validatDataset: True  # "True" default value
BuildDataset:
  useMultiThreading: True  # "True" default value

shhs-path: D:/datasets/SHHS/
mros-path: D:/datasets/MrOs/mros/
mesa-path: D:/datasets/MESA/
tsm-path: D:/datasets/TSM/records

Local execution

Requirements

• Python >= 3.5
• PyTorch
• packages in requirements.txt: pip install -r requirements.txt

Train a Model

There are default Configs in the folder configs/datasets for each dataset. The file configs/datasets/[dataset identifier]/init.yaml loads the default values to execute further commands.

To test the Training process, you can execute the training only using few recordings:

phases run -c userconfigs/local.yaml,configs/datasets/shhs/init.yaml,configs/datasets/shhs/version-debug.yaml Training

If phases is not installed in your path you can use python -m phases ... instead.

This command starts the extraction of the dataset and the training process.

Docker

Docker compose

version: "3.7"

services:
  apptorch:
    image: registry.gitlab.com/tud.ibmt.public/pyphases/arousaldetector/torch
    entrypoint: python -m phases
    volumes:
      - ./data:/app/data
      - ./logs:/app/logs
      # you can use this to reference your local datasets
      - ./datasets/:/app/datasets
      # just a placeholder for your custom configs
      - ./userconfigs:/app/userconfigs
    # required for gpu execution using a nvidia gpu
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              device_ids: [ '0' ]
              capabilities: [ gpu ]

Create a local config file in userconfigs/local.yml and add the following content:

useLoader: shhs
shhs-path: /app/datasets/SHHS1

Now you can test the project using docker compose:

docker compose run apptorch -c userconfigs/local.yml run Training

Run the complete Training:

docker compose run apptorch -c userconfigs/local.yml,configs/shhs.predefined.yaml run Training

Usage

Create Dataset

Create a new Dataset using

# config.yml
# setup shhs

shhs-path: ./datasets/shhs
useLoader: shhs

dataversion:
  version: All
  groupBy: Null
  recordIds: Null

# export specification
export:
  annotations:
    - apnea
    - arousal
    - limb
    - sleepStage
    - light
  channels:
    - [EEG]
    - [EEG Ch2]
    - [EOG(L), EOG LOC-A2, E1-M2, "E1:M2"]
    - [EOG(R), EOG ROC-A1, "E2:M1"]
    - [EMG, EMG Chin, Chin1-Chin2, Chin]
    - [POSITION, Body]
    - [THOR RES, Effort THO, CHEST, RIP Thora]
    - [ABDO RES, Effort ABD, ABD, RIP Abdom]
    - [SaO2, SpO2]
    - [FLOW]

single record

phases run --set recordId=shhs1-200001 -c config.yml Export

full dataset

phases run -c config.yml Export

Define Custom Data Loading

Development

Change dataset

make sure the right paths are set in project.yaml to use the dataset.

phases run -c configs/datasets/[DATASET]/init.yml run Training

for example:

phases run -c configs/datasets/mesa/init.yml run Training

Change Model

Add new metric to scoring

Adding a new metric for the validation/evaluation you register a new metric using the Scorer class from pyPhasesML.

from pyPhasesML import Scorer
# ...
class Setup(Phase):
    def prepareConfig(self):
        # ...

        def scoreRecord(truth, prediction):
            # calculate the score for single record
            return 1.0

        def scoreAllSamples(truth, prediction):
            # calculate the score for all samples
            return 1.0

        Scorer.registerMetric("score1", scoreRecord, scoreAllSamples)

Update config:

# add metric to training validation

trainingParameter:
  validationMetrics:
    - accuracy
    - score1
# add metric to evaluation

eval:
  metrics:
  - kappa
  - score1

Run Tests

phases test test

Using docker: docker compose run --rm phases-test test test -v

Create debug docker container

docker compose run --rm --entrypoint bash phases-test

pyPhase structure

Complete	Training with pyPhases

Apnea

Use all channels: configs/datasets/shhs/channels.yaml Change Classification: configs/aasm-apnea.yaml Use Hotencoding for model inputs: configs/hotEncode.yaml Change Model: configs/models/apneaunet.yaml

Training: phases run -c userconfigs/local.yml,configs/datasets/shhs/init.yaml,configs/datasets/shhs/version-debug.yaml,configs/datasets/shhs/channels.yaml,configs/aasm-apnea.yaml,configs/hotEncode.yaml,configs/models/apneaunet.yaml Training