more-metrics 0.11.0

A Python package for computing additional classification metrics for machine learning or AI models. These metrics include and also known as AUC, NRI, and IDI

AUC_NRI_IDI_python_functions

Custom python functions to help you further analyse machine learning models and diagnostic test.

Will help you make plots and compute evaluation metrics as seen in Nature Article, Leong et al. 2021

Metrics to compute and plot:

• AUC = Area Under the Curve
• NRI = Net Reclassification Index
• IDI = Integrated Discrimination Improvement
• Functions to compute bootstrap p-values for AUC and NRI differences

Run "example.ipynb" Jupyter notebook to see and use functions

Installation

pip install -r requirements.txt
pip install .

Running tests

pytest -q

Formulas

AUC [AUC = \int_0^1 TPR(FPR), dFPR ]

NRI [NRI = (P_{\text{up}|\text{event}} - P_{\text{down}|\text{event}}) + (P_{\text{down}|\text{non-event}} - P_{\text{up}|\text{non-event}})]

IDI [IDI = (\bar{p}{\text{new},1} - \bar{p}{\text{ref},1}) - (\bar{p}{\text{new},0} - \bar{p}{\text{ref},0})]

Usage

Import the package and call any metric helpers. See the example notebook for a detailed walkthrough.

Code and concepts further explained in the following post: "Area Under the Curve and Beyond" or "On Medium/Towards Data Science"

---

contact Lambert Leong