cerebralcortex-kernel 3.3.16

Backend data analytics platform for MD2K software

CerebralCortex Kernel

Cerebral Cortex is the big data cloud companion of mCerebrum designed to support population-scale data analysis, visualization, model development, and intervention design for mobile sensor data.

You can find more information about MD2K software on our software website or the MD2K organization on our MD2K website.

CerebralCortex Kernel is part of our CerebralCortex cloud platform. CerebralCortex-Kernel is capable of parallelizing tasks and scale a job to n-number of cores/machines. CerebralCortex Kernel offers some builtin features as follows:

Installation

Dependencies

CerebralCortex Kernel requires java 8 to run. Java 8 prior to version 8u92 support is deprecated as of CerebralCortex-Kernel 3.3.0.

• check java version - java -version
• set JAVA_HOME to java 8
• OR start python shell with JAVA_HOME=/path/to/java/Home python3

Install using pip

CerebralCortex-Kernel requires minimum Python3.6. To install CerebralCortex-Kernel as an API:

pip3 install cerebralcortex-kernel

• Note: please use appropriate pip (e.g., pip, pip3, pip3.6 etc.) installed on your machine

Install from source code

• Clone repo - git clone https://github.com/MD2Korg/CerebralCortex-Kernel.git
• cd CerebralCortex-Kernel
• python3 setup.py install

Usage

from cerebralcortex.kernel import Kernel
CC = Kernel(cc_configs="default")

# to view default configs
print(CC.config)

# default data storage path is
# /user/home/folder/cc_data

By default Kernel will load default configs. Please have a look at all available configurations for CerebralCortex-Kernel. You may also load config files as:

CC = Kernel(configs_dir_path="dir/path/to/configs/", new_study=True)

How to use builtin algorithms

Using builtin algorithms are as easy as loading data, passing it to algorithm and get the results. Below is an example on how to compute CGM Glucose Variability Metrics.

• Download Glucose Data. The device used to collect glucose was the Dexcom G6 Continuous Glucose Monitor
• Install Cerebral Cortex Kernel pip install cerebralcortex-kernel
• Open terminal and start python3 shell

Python Code

# import packages
from cerebralcortex.kernel import Kernel
from cerebralcortex.algorithms.glucose.glucose_variability_metrics import glucose_var

# Create Kernel object
CC = Kernel(cc_configs="default", new_study=True)

# Read sample CSV data
ds = CC.read_csv("/path/of/the/downloaded/file/sample.csv", stream_name="cgm_glucose_variability_metrics", header=True)

# view sample data
ds.show(2)

# Apply glucose_variability_metrics algorithm on the data
results = glucose_var(ds)

# view results
results.show(2)

# save computed data 
CC.save_stream(results)

Please have a look at jupyter notebook for basic operation that could be perform on DataStream object.

Algorithms to Analyze Sensor Data

External CerebralCortex-Kernel offers following builtin algorithms to analyze sensor data.

• ECG sensor data quality
• ECG RR Interval Computation
• Heart Rate Variability Feature Computation
• CGM Glucose Variability Metrics
• GPS Data Clustering
• Sensor Data Interpolation
• Statistical Features Computation
• List of all available algorithms

Markers with ML Models

• Stress Detection using ECG data
• mContain Social Crowding
• Brushing Detection using Accelerometer and Gyro Data (TODO)

Visualization

• Basic Plots for Timeseries Data
• Plot GPS Clusters on Map
• Stress Visualization

Import and Document Data

• Import CSV Data in CerebralCortex-Kernel Format
• Document imported Data using MetaData Module

External CerebralCortex-Kernel Supported Platforms

• mProv
• mFlow

Examples

• Jupyter Notebooks

Documentation

• Source code documentation

Deploy on Cloud

CerebralCortex-Kernel is a part of CerebralCortex cloud platform. To test the complete cloud platform, please visit CerebralCortex.

FAQ

1 - Do I need whole CerebralCortex cloud platform to use CerebralCortex-Kernal?

No! If you want to use CerebralCortex-Kernel independently.

2 - How can I change NoSQL backend storage layer?

CerebralCortex-Kernel follows component based structure. This makes it easier to add/remove features.

• Add a new class in Data manager-Raw.
• New class must have read/write methods. Here is a sample skeleton class with mandatory methods required in the new class.
• Create an object of new class in Data-Raw with appropriate parameters.
• Add appropriate configurations in cerebralcortex.yml in (NoSQL Storage)[https://github.com/MD2Korg/CerebralCortex-Kernel/blob/master/conf/cerebralcortex.yml#L8] section.

3 - How can I replace MySQL with another SQL storage system?

• Add a new class in Data manager-SQL.
• New class must implement all of the methods available in stream_handler.py class.
• Create an object of new class in Data-SQL with appropriate parameters.
• Add appropriate configurations in cerebralcortex.yml in Relational Storage section.

4 - Where are all the backend storage related classes/methods?

In Data manager-Raw. You can add/change any backend storage.

Contributing

Please read our Contributing Guidelines for details on the process for submitting pull requests to us.

We use the Python PEP 8 Style Guide.

Our Code of Conduct is the Contributor Covenant.

Bug reports can be submitted through JIRA.

Our discussion forum can be found here.

Versioning

We use Semantic Versioning for versioning the software which is based on the following guidelines.

MAJOR.MINOR.PATCH (example: 3.0.12)

1. MAJOR version when incompatible API changes are made,
2. MINOR version when functionality is added in a backwards-compatible manner, and
3. PATCH version when backwards-compatible bug fixes are introduced.

For the versions available, see this repository's tags.

Contributors

Link to the list of contributors who participated in this project.

License

This project is licensed under the BSD 2-Clause - see the license file for details.

Acknowledgments

• National Institutes of Health - Big Data to Knowledge Initiative
  • Grants: R01MD010362, 1UG1DA04030901, 1U54EB020404, 1R01CA190329, 1R01DE02524, R00MD010468, 3UH2DA041713, 10555SC
• National Science Foundation
  • Grants: 1640813, 1722646
• Intelligence Advanced Research Projects Activity
  • Contract: 2017-17042800006