metriculous
Measure, visualize, and compare machine learning model performance without the usual boilerplate. Breaking API improvements to be expected.
Installation
$ pip install metriculous
Or, for the latest unreleased version:
$ pip install git+https://github.com/metriculous-ml/metriculous.git
Comparing Regression Models
Click to see more code
import numpy as np
# Mock the ground truth, a one-dimensional array of floats
ground_truth = np.random.random(300)
# Mock the output of a few models
perfect_model = ground_truth
noisy_model = ground_truth + 0.1 * np.random.randn(*ground_truth.shape)
random_model = np.random.randn(*ground_truth.shape)
zero_model = np.zeros_like(ground_truth)
import metriculous
metriculous.compare_regressors(
ground_truth=ground_truth,
model_predictions=[perfect_model, noisy_model, random_model, zero_model],
model_names=["Perfect Model", "Noisy Model", "Random Model", "Zero Model"],
).save_html("comparison.html").display()
This will save an HTML file with common regression metrics and charts, and if you are working in a Jupyter notebook will display the output right in front of you:
Comparing Classification Models
Click to see more code
import numpy as np
def normalize(array2d: np.ndarray) -> np.ndarray:
return array2d / array2d.sum(axis=1, keepdims=True)
class_names = ["Cat", "Dog", "Pig"]
num_classes = len(class_names)
num_samples = 500
# Mock ground truth
ground_truth = np.random.choice(range(num_classes), size=num_samples, p=[0.5, 0.4, 0.1])
# Mock model predictions
perfect_model = np.eye(num_classes)[ground_truth]
noisy_model = normalize(
perfect_model + 2 * np.random.random((num_samples, num_classes))
)
random_model = normalize(np.random.random((num_samples, num_classes)))
import metriculous
metriculous.compare_classifiers(
ground_truth=ground_truth,
model_predictions=[perfect_model, noisy_model, random_model],
model_names=["Perfect Model", "Noisy Model", "Random Model"],
class_names=class_names,
one_vs_all_figures=True,
).display()
Development
Poetry
This project uses poetry to manage
dependencies. Please make sure it is installed for the required python version. Then install the dependencies with poetry install.
Makefile
A Makefile is used to automate common development workflows. Type make or make help to see a list of available commands. Before commiting changes it is recommended to run make format check test.
3 maintainers
Metadata
3 maintainers
Release files for metriculous 0.3.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| metriculous-0.3.0.tar.gz | 40.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| metriculous-0.3.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 91.5 kB
Release files / metriculous-0.3.0.tar.gz
| Download URL | metriculous-0.3.0.tar.gz |
|---|---|
| Size | 40.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6bce4e4746e707517fa3bcb3f7307f9c71a267566bcbddaa1257194db4e2d741
|
|
BLAKE2b-256 checksum How to use checksums |
645ddae8ff38946fa2a97463b36e3b01fb908f650437c20d2f0c3ceaa6c52b94
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.1.4 CPython/3.7.8 Darwin/19.6.0
|
Release files / metriculous-0.3.0-py3-none-any.whl
| Download URL | metriculous-0.3.0-py3-none-any.whl |
|---|---|
| Size | 50.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
4f6116596facc988f0cde6766a9596e824cab1992849ab04a07339bc066d12f0
|
|
BLAKE2b-256 checksum How to use checksums |
b4a620832436e207d8459f577d6b86e15c3d785fc2642f8eb1cadeb39fc34cb1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.1.4 CPython/3.7.8 Darwin/19.6.0
|
