Python package to measure the calibration of probabilistic classifiers
Project description
classifier-calibration
Example Usage
import sklearn.linear_model
import sklearn.ensemble
import pandas as pd
import numpy as np
from calibr8 import classwise_ece
Binary classification
X, Y = sklearn.datasets.make_classification(n_samples=100000,n_informative=6,n_classes=2)
clf = sklearn.linear_model.LogisticRegression().fit(X,Y)
predicted_probabilities = clf.predict_proba(X)
calibration_loss = classwise_ece(pred_probs=predicted_probabilities,labels=Y)
print(calibration_loss)
Multi-class classification
We implement and compare calibration and accuracy of random forest and logistic regression classifiers for the 3-class classification problem
# Make classification data
num_classes = 3
X, Y = sklearn.datasets.make_classification(n_samples=100000,n_informative=6,n_classes=num_classes)
lr = sklearn.linear_model.LogisticRegression().fit(X,Y)
rf = sklearn.ensemble.RandomForestClassifier().fit(X,Y)
lr_predicted_probabilities = lr.predict_proba(X)
rf_predicted_probabilities = rf.predict_proba(X)
lr_calibration_loss, lr_bin_weights = classwise_ece(pred_probs=lr_predicted_probabilities,labels=Y,return_weights=True)
rf_calibration_loss, rf_bin_weights = classwise_ece(pred_probs=rf_predicted_probabilities,labels=Y,return_weights=True)
print(round(100*lr_calibration_loss,2),'%',round(100*rf_calibration_loss,2),'%')
# print distribution of weights across bins for each class' set of predictions
for k in range(num_classes):
print('Logistic regression bin weights for class',k,'predictions:')
print(lr_bin_weights[k])
print('Random forest bin weights for class',k,'predictions:')
print(rf_bin_weights[k])
print()
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for classifier_calibration-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8f8f8b567c15d7f1fb54fa3364f313e1d6e3717c18eead48aae4719d410560e5 |
|
MD5 | 1ebb4c5b01c57a513e776ff9649f4b7c |
|
BLAKE2b-256 | c0d5a669ffa5affd5217b8a64176587941c85c1d4945faf3eca5dd5a2bf9b97f |
Close
Hashes for classifier_calibration-0.1.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9dd995a4a6b837b1a2d9d0fca9bd85f104682f814dea0c72803e09d0b14b9956 |
|
MD5 | 909a6679f657d5c82cada2967f0d344e |
|
BLAKE2b-256 | 8cdc4768b357887a7c4758b9a8594d874a510cb98c86def13228607e16f8afe2 |