Skip to main content

Very unstable library containing utilities to measure and visualize statistical properties of machine learning models.

Project description

Launch Binder Current GitHub Actions build status Checked with mypy PyPI version PyPI - Python Version License MIT Friends with Luminovo.AI

metriculous

Unstable python library with utilities to measure, visualize and compare statistical properties of machine learning models. Breaking improvements to be expected.

Installation

$ pip install metriculous

Or, for the latest unreleased version:

$ pip install git+https://github.com/metriculous-ml/metriculous.git

Or, to avoid getting surprised by breaking changes:

$ pip install git+https://github.com/metriculous-ml/metriculous.git@YourFavoriteCommit

Usage

Comparing Regression Models Binder

import numpy as np

# Mock the ground truth, a one-dimensional array of floats
ground_truth = np.random.random(300)

# Mock the output of a few models
perfect_model = ground_truth
noisy_model = ground_truth + 0.1 * np.random.randn(*ground_truth.shape)
random_model = np.random.randn(*ground_truth.shape)
zero_model = np.zeros_like(ground_truth)

import metriculous

metriculous.compare_regressors(
    ground_truth=ground_truth,
    model_predictions=[perfect_model, noisy_model, random_model, zero_model],
    model_names=["Perfect Model", "Noisy Model", "Random Model", "Zero Model"],
).save_html("comparison.html").display()

This will save an HTML file with common regression metrics and charts, and if you are working in a Jupyter notebook will display the output right in front of you:

Screenshot of Metriculous Regression Metrics Screenshot of Metriculous Regression Figures

Comparing Classification Models Binder

For an example that evaluates and compares classifiers, please refer to the quickstart notebook for classification.

Development

Poetry

This project uses poetry to manage dependencies. Please make sure it is installed for the required python version. Then install the dependencies with poetry install.

Makefile

A Makefile is used to automate common development workflows. Type make or make help to see a list of available commands. Before commiting changes it is recommended to run make format check test.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

metriculous-0.2.1.tar.gz (37.5 kB view hashes)

Uploaded Source

Built Distribution

metriculous-0.2.1-py3-none-any.whl (47.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page