e2epyppg 0.0.9

An End-to-End PPG processing pipeline: from quality assessment and motion artifacts romval to HR/HRV extraction

End-to-End PPG Processing Pipeline (e2epyppg)

Welcome to the PPG Signal Processing Pipeline, a comprehensive package designed for extracting accurate Heart Rate (HR) and Heart Rate Variability (HRV) data from Photoplethysmogram (PPG) signals.

Introduction

This project provides a robust pipeline for processing PPG signals, extracting reliable HR and HRV parameters. The pipeline encompasses various stages, including filtering, signal quality assessment (SQA), signal reconstruction, peak detection and interbeat interval (IBI) extraction, and HR and HRV computation.

Preprocessing

The input raw PPG signal undergoes filtering to remove undesired frequencies. A second-order Butterworth bandpass filter is employed, allowing frequencies within a specific range to pass while rejecting frequencies outside.

Signal Quality Assessment (SQA)

SQA involves identifying clean and noisy parts within PPG signals. Our SQA approach requires PPG signals in a fixed length, which necessitates segmenting the input signals. To this end, we apply a moving window segmentation technique, where the PPG signals are divided into overlapping segments, each spanning 30 seconds, by sliding a window over the signal. The SQA process includes PPG feature extraction and classification, employing a one-class support vector machine model to distinguish between "Reliable" and "Unreliable" segments.

Noise Reconstruction

Noisy parts within PPG signals, shorter than a specified threshold, are reconstructed using a deep convolutional generative adversarial network (GAN). The GAN model includes a generator trained to produce synthetic clean PPG segments and a discriminator to differentiate real from synthetic signals. The reconstruction process is applied iteratively to ensure denoising.

Peak Detection and IBI Extraction

Systolic peaks in PPG signals are identified using a deep-learning-based method with dilated Convolutional Neural Networks (CNN) architecture. PPG peak detection enables the extraction of IBI values that serve as the basis for obtaining HR and HRV. IBI represents the time duration between two consecutive heartbeats and is computed by measuring the time interval between systolic peaks within the PPG signals.

HR and HRV Extraction

HR and HRV parameters are computed from the extracted IBIs. A variety of metrics are calculated, including:

• HR: Heart rate
• MeanNN: The mean of the RR intervals
• SDNN: The standard deviation of the RR intervals
• SDANN: The standard deviation of average RR intervals
• SDNNI: The mean of the standard deviations of RR intervals
• RMSSD: The square root of the mean of the squared successive differences between adjacent RR intervals
• SDSD: The standard deviation of the successive differences between RR intervals
• CVNN: The standard deviation of the RR intervals (SDNN) divided by the mean of the RR intervals (MeanNN)
• CVSD: The root mean square of successive differences (RMSSD) divided by the mean of the RR intervals (MeanNN)
• MedianNN: The median of the RR intervals
• MadNN: The median absolute deviation of the RR interval
• MCVNN: The median absolute deviation of the RR intervals (MadNN) divided by the median of the RR intervals (MedianNN)
• IQRNN: The interquartile range (IQR) of the RR intervals
• Prc20NN: The 20th percentile of the RR intervals
• Prc80NN: The 80th percentile of the RR intervals
• pNN50: The proportion of RR intervals greater than 50ms, out of the total number of RR intervals
• pNN20: The proportion of RR intervals greater than 20ms, out of the total number of RR intervals
• MinNN: The minimum of the RR intervals
• MaxNN: The maximum of the RR intervals
• TINN: A geometrical parameter of the HRV, or more specifically, the baseline width of the RR intervals distribution obtained by triangular interpolation, where the error of least squares determines the triangle. It is an approximation of the RR interval distribution
• HTI: The HRV triangular index, measuring the total number of RR intervals divided by the height of the RR intervals histogram
• ULF: The spectral power of ultra low frequencies (by default, .0 to .0033 Hz)
• VLF: The spectral power of very low frequencies (by default, .0033 to .04 Hz)
• LF: The spectral power of low frequencies (by default, .04 to .15 Hz)
• HF: The spectral power of high frequencies (by default, .15 to .4 Hz)
• VHF: The spectral power of very high frequencies (by default, .4 to .5 Hz)
• LFHF: The ratio obtained by dividing the low frequency power by the high frequency power
• LFn: The normalized low frequency, obtained by dividing the low frequency power by the total power
• HFn: The normalized high frequency, obtained by dividing the low frequency power by the total power
• LnHF: The log transformed HF
• SD1: Standard deviation perpendicular to the line of identity
• SD2: Standard deviation along the identity line
• SD1/SD2: ratio of SD1 to SD2. Describes the ratio of short term to long term variations in HRV
• S: Area of ellipse described by SD1 and SD2 (pi * SD1 * SD2)

Installation

Install the e2epyppg package using the following pip command:

pip install e2epyppg

Usage

1. End-to-End HR and HRV Extraction Function

e2e_hrv_extraction()

End-to-end HR and HRV extraction from an input raw PPG signal.

Args:

• input_sig (np.ndarray): The input PPG signal.
• sampling_rate (int): The sampling rate of the input signal.
• window_length_sec (int, optional): The desired window length for HR and HRV extraction in seconds (default: 60).
• peak_detection_method (str, optional): The method used for peak detection. Valid inputs: 'kazemi', 'nk' (Neurokit), and 'heartpy' (default: 'kazemi').

Returns:

• hrv_data (pd.DataFrame): A DataFrame containing HRV parameters ('None' if no clean segments detected in the signal).

Note:

If it takes a long time to process, consider segmenting the signal into shorter segments for more efficient processing.

Reference:

Feli, M., Kazemi, K., Azimi, I., Wang, Y., Rahmani, A., & Liljeberg, P. (2023). End-to-End PPG Processing Pipeline for Wearables: From Quality Assessment and Motion Artifacts Removal to HR/HRV Feature Extraction. In 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE.

Usage Example:

from e2epyppg.utils import get_data
from e2epyppg.e2e_ppg_pipeline import e2e_hrv_extraction

# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)
file_name = "201902020222_Data.csv"
input_sig = get_data(file_name=file_name)
sampling_rate = 20

# Set the desired window length for HR and HRV features extraction (in seconds)
window_length_sec = 60

# Set the peak detection method (optional)
peak_detection_method = 'kazemi'

# Call the end-to-end function
hrv_data = e2e_hrv_extraction(input_sig, sampling_rate, window_length_sec, peak_detection_method)

2. PPG Signal Quality Assessment (SQA) Function

sqa()

Perform PPG Signal Quality Assessment (SQA).

This function assesses the quality of a PPG signal by classifying its segments as reliable (clean) or unreliable (noisy) using a pre-trained model. The clean indices represent parts of the PPG signal that are deemed reliable, while the noisy indices indicate parts that may be affected by noise or artifacts.

Args:

• sig (np.ndarray): PPG signal.
• sampling_rate (int): Sampling rate of the PPG signal.
• filter_signal (bool, optional): True if the signal has not been filtered using a bandpass filter.

Returns:

• clean_indices (list): A list of clean indices.
• noisy_indices (list): A list of noisy indices.

Note:

If it takes a long time to process, consider segmenting the signal into shorter segments for more efficient processing.

Reference:

Feli, M., Azimi, I., Anzanpour, A., Rahmani, A. M., & Liljeberg, P. (2023). An energy-efficient semi-supervised approach for on-device photoplethysmogram signal quality assessment. Smart Health, 28, 100390.

Usage Example:

from e2epyppg.utils import get_data
from e2epyppg.ppg_sqa import sqa

# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)
file_name = "201902020222_Data.csv"
input_sig = get_data(file_name=file_name)
sampling_rate = 20

# Set this parameter True if the signal has not been filtered:
filter_signal = True

# Call the PPG signal quality assessment function
clean_indices, noisy_indices = sqa(input_sig, sampling_rate, filter_signal)

3. PPG Signal Reconstruction Function

reconstruction()

Reconstruct noisy PPG signals using Generative Adversarial Network (GAN).

Args:

• sig (np.ndarray): Original PPG signal.
• clean_indices (list): List of indices representing clean parts (obtained from sqa()).
• noisy_indices (list): List of indices representing noisy parts (obtained from sqa()).
• sampling_rate (int): Sampling rate of the signal.
• filter_signal (bool, optional): True if the signal has not been filtered using a bandpass filter.

Returns:

• ppg_signal (np.ndarray): Reconstructed PPG signal (if reconstruction is applied; otherwise, returns the original signal).
• clean_indices (list): Updated indices of clean parts (if reconstruction is applied; otherwise, returns the original indices of clean parts).
• noisy_indices (list): Updated indices of noisy parts (if reconstruction is applied; otherwise, returns the original indices of noisy parts).

Note:

If it takes a long time to process, consider segmenting the signal into shorter segments for more efficient processing.

Reference:

Wang, Y., Azimi, I., Kazemi, K., Rahmani, A. M., & Liljeberg, P. (2022, July). Ppg signal reconstruction using deep convolutional generative adversarial network. In 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 3387-3391). IEEE.

Usage Example:

from e2epyppg.utils import get_data
from e2epyppg.ppg_sqa import sqa
from e2epyppg.ppg_reconstruction import reconstruction

# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)
file_name = "201902020222_Data.csv"
input_sig = get_data(file_name=file_name)
sampling_rate = 20

# Set this parameter True if the signal has not been filtered:
filter_signal = True

# Call the PPG signal quality assessment function
clean_indices, noisy_indices = sqa(input_sig, sampling_rate, filter_signal)

# Call the PPG reconstruction function
ppg_reconstructed, updated_clean_indices, updated_noisy_indices = reconstruction(input_sig, clean_indices, noisy_indices, sampling_rate, filter_signal)

4. Clean Segment Extraction Function

clean_seg_extraction()

Scan the clean parts of the signal and extract clean segments based on the input window length.

Args:

• sig (np.ndarray): Input PPG signal.
• noisy_indices (list): List of noisy segment indices (obtained from sqa() or reconstruction()).
• window_length (int): Desired window length for clean segment extraction in terms of samples (not seconds).

Returns:

• clean_segments (list): List of clean PPG segments with the specified window length and their starting index.

Usage Example:

from e2epyppg.utils import get_data
from e2epyppg.ppg_sqa import sqa
from e2epyppg.ppg_reconstruction import reconstruction
from e2epyppg.ppg_clean_extraction import clean_seg_extraction

# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)
file_name = "201902020222_Data.csv"
input_sig = get_data(file_name=file_name)
sampling_rate = 20
window_length_sec = 60

# Set this parameter True if the signal has not been filtered:
filter_signal = True

# Call the PPG signal quality assessment function
clean_indices, noisy_indices = sqa(input_sig, sampling_rate, filter_signal)

# Call the PPG reconstruction function
ppg_reconstructed, updated_clean_indices, updated_noisy_indices = reconstruction(input_sig, clean_indices, noisy_indices, sampling_rate, filter_signal)

# Set the window length for HR and HRV extraction in terms of samples
window_length = window_length_sec*sampling_rate

# Call the clean segment extraction function
clean_segments = clean_seg_extraction(ppg_reconstructed, updated_noisy_indices, window_length)

5. Peak Detection Function

peak_detection()

Detect peaks in clean PPG segments using the specified peak detection method.

Args:

• clean_segments (list): List of clean PPG segments with the specified window length and their starting index (obtained from clean_seg_extraction()).
• sampling_rate (int): Sampling rate of the PPG signal.
• method (str, optional): Peak detection method. Valid inputs: 'kazemi', 'nk' (Neurokit), and 'heartpy'. (default: 'kazemi').

Returns:

• total_peaks (list): List of lists, each containing the detected peaks for a corresponding clean segment.

References:

• Kazemi method: Kazemi, K., Laitala, J., Azimi, I., Liljeberg, P., & Rahmani, A. M. (2022). Robust ppg peak detection using dilated convolutional neural networks. Sensors, 22(16), 6054.
• Neurokit method: Makowski, D., Pham, T., Lau, Z. J., Brammer, J. C., Lespinasse, F., Pham, H., ... & Chen, S. A. (2021). NeuroKit2: A Python toolbox for neurophysiological signal processing. Behavior research methods, 1-8.
• HeartPY method: Van Gent, P., Farah, H., Nes, N., & van Arem, B. (2018, June). Heart rate analysis for human factors: Development and validation of an open source toolkit for noisy naturalistic heart rate data. In Proceedings of the 6th HUMANIST Conference (pp. 173-178).

Usage Example:

from e2epyppg.utils import get_data
from e2epyppg.ppg_sqa import sqa
from e2epyppg.ppg_reconstruction import reconstruction
from e2epyppg.ppg_clean_extraction import clean_seg_extraction
from e2epyppg.ppg_peak_detection import peak_detection

# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)
file_name = "201902020222_Data.csv"
input_sig = get_data(file_name=file_name)
sampling_rate = 20
window_length_sec = 60

# Set this parameter True if the signal has not been filtered:
filter_signal = True

# Call the PPG signal quality assessment function
clean_indices, noisy_indices = sqa(input_sig, sampling_rate, filter_signal)

# Call the PPG reconstruction function
ppg_reconstructed, updated_clean_indices, updated_noisy_indices = reconstruction(input_sig, clean_indices, noisy_indices, sampling_rate, filter_signal)

# Set the window length for HR and HRV extraction in terms of samples
window_length = window_length_sec*sampling_rate

# Call the clean segment extraction function
clean_segments = clean_seg_extraction(ppg_reconstructed, updated_noisy_indices, window_length)

# Set the peak detection method (optional)
peak_detection_method = 'kazemi'

# Call the peak detection function
peaks = peak_detection(clean_segments, sampling_rate, peak_detection_method)

6. HR and HRV Extraction Function

hrv_extraction()

Calculate HR and HRV parameters from clean segments peak indices.

Args:

• clean_segments (list): List of clean PPG segments and their starting index (obtained from clean_seg_extraction()).
• peaks (list): List of lists, each containing the detected peaks for a corresponding clean segment (obtained from peak_detection()).
• sampling_rate (int): Sampling rate of the PPG signal.
• window_length (int): Desired window length for HR and HRV extraction in terms of samples (not seconds).

Returns:

• hrv_data (pd.DataFrame): DataFrame containing HRV parameters for all clean segments.

Usage Example:

from e2epyppg.utils import get_data
from e2epyppg.ppg_sqa import sqa
from e2epyppg.ppg_reconstruction import reconstruction
from e2epyppg.ppg_clean_extraction import clean_seg_extraction
from e2epyppg.ppg_peak_detection import peak_detection
from e2epyppg.ppg_hrv_extraction import hrv_extraction

# Provide your PPG signal and sampling rate (you can use your own signal in format `np.ndarray`)
file_name = "201902020222_Data.csv"
input_sig = get_data(file_name=file_name)
sampling_rate = 20
window_length_sec = 60

# Set this parameter True if the signal has not been filtered:
filter_signal = True

# Call the PPG signal quality assessment function
clean_indices, noisy_indices = sqa(input_sig, sampling_rate, filter_signal)

# Call the PPG reconstruction function
ppg_reconstructed, updated_clean_indices, updated_noisy_indices = reconstruction(input_sig, clean_indices, noisy_indices, sampling_rate, filter_signal)

# Set the window length for HR and HRV extraction in terms of samples
window_length = window_length_sec*sampling_rate

# Call the clean segment extraction function
clean_segments = clean_seg_extraction(ppg_reconstructed, updated_noisy_indices, window_length)

# Set the peak detection method (optional)
peak_detection_method = 'kazemi'

# Call the peak detection function
peaks = peak_detection(clean_segments, sampling_rate, peak_detection_method)

# Call the HR and HRV feature extraction function
hrv_data = hrv_extraction(clean_segments, peaks, sampling_rate, window_length)

Citation

If you use our work in your research, please consider citing the following papers:

1. End-to-End PPG Processing Pipeline for Wearables: From Quality Assessment and Motion Artifacts Removal to HR/HRV Feature Extraction (PPG Pipeline Paper)

   • Conference: 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
   • Authors: Mohammad Feli, Kianoosh Kazemi, Iman Azimi, Yuning Wang, Amir Rahmani, Pasi Liljeberg

   • @inproceedings{feli2023end,
       title={End-to-End PPG Processing Pipeline for Wearables: From Quality Assessment and Motion Artifacts Removal to HR/HRV Feature Extraction},
       author={Feli, Mohammad and Kazemi, Kianoosh and Azimi, Iman and Wang, Yuning and Rahmani, Amir and Liljeberg, Pasi},
       booktitle={2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)},
       year={2023},
       organization={IEEE}
     }
     

2. An Energy-Efficient Semi-Supervised Approach for On-Device Photoplethysmogram Signal Quality Assessment (PPG Signal Quality Assessment Paper)

   • Journal: Smart Health
   • Authors: Mohammad Feli, Iman Azimi, Arman Anzanpour, Amir M Rahmani, Pasi Liljeberg

   • @article{feli2023energy,
       title={An Energy-Efficient Semi-Supervised Approach for On-Device Photoplethysmogram Signal Quality Assessment},
       author={Feli, Mohammad and Azimi, Iman and Anzanpour, Arman and Rahmani, Amir M and Liljeberg, Pasi},
       journal={Smart Health},
       volume={28},
       pages={100390},
       year={2023},
       publisher={Elsevier}
     }
     

3. PPG Signal Reconstruction Using Deep Convolutional Generative Adversarial Network (PPG Reconstruction Paper)

   • Conference: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)
   • Authors: Yuning Wang, Iman Azimi, Kianoosh Kazemi, Amir M Rahmani, Pasi Liljeberg

   • @inproceedings{wang2022ppg,
       title={PPG Signal Reconstruction Using Deep Convolutional Generative Adversarial Network},
       author={Wang, Yuning and Azimi, Iman and Kazemi, Kianoosh and Rahmani, Amir M and Liljeberg, Pasi},
       booktitle={2022 44th Annual International Conference of the IEEE Engineering in Medicine \& Biology Society (EMBC)},
       pages={3387--3391},
       year={2022},
       organization={IEEE}
     }
     

4. Robust PPG Peak Detection Using Dilated Convolutional Neural Networks (PPG Peak Detection Paper)

   • Journal: Sensors
   • Authors: Kianoosh Kazemi, Juho Laitala, Iman Azimi, Pasi Liljeberg, Amir M Rahmani

   • @article{kazemi2022robust,
       title={Robust PPG Peak Detection Using Dilated Convolutional Neural Networks},
       author={Kazemi, Kianoosh and Laitala, Juho and Azimi, Iman and Liljeberg, Pasi and Rahmani, Amir M},
       journal={Sensors},
       volume={22},
       number={16},
       pages={6054},
       year={2022},
       publisher={MDPI}
     }
     

Acknowledgments

We would like to express our sincere gratitude to Prof Pasi Liljeberg and Prof Amir M. Rahmani for their guidance and support throughout this project.

Contributing

We welcome contributions to enhance and extend the capabilities of the PPG Signal Processing Pipeline.

License

This project is licensed under the terms of the MIT License.