nannyml 0.3.1

The NannyML library, monitoring model performance since 2020.

Website • Docs • Community Slack

Basic overview

NannyML helps you monitor your ML models in production by:

• estimating performance in absence of ground truth
• calculating realized performance metrics
• detecting data drift on model inputs, model outputs and targets

Installing the latest stable release

pip install nannyml

Installing the latest development version

python -m pip install git+https://github.com/NannyML/nannyml

Getting started

import nannyml as nml
import pandas as pd

# Load some data
reference_data, analysis_data, _ = nml.load_synthetic_sample()
data = pd.concat([reference_data, analysis_data])
metadata = nml.extract_metadata(reference_data)

# Estimate performance
estimator = nml.CBPE(metadata).fit(reference_data)
estimates = estimator.estimate(data)

estimates.plot(kind='performance').show()

Examples

• Performance estimation
• Realized performance calculation
• Univariate model input drift detection
• Multivariate model input drift detection
• Model output drift detection
• Model target distribution

Development setup

• Read the docs on how to contribute