Tremetrics

Tremendous Metrics.

Installation

You can install Tremetrics from PyPi using pip.

pip install tremetrics

Usage

ConfusionMatrix

from tremetrics import ConfusionMatrix

y_true, y_pred = ...                            # Generate predictions
cm = ConfusionMatrix.from_pred(y_true, y_pred)  # Create a new confusion matrix object

print(cm)                                       # Print the confusion matrix
array_for_further_use = cm.array                # Get the matrix as a numpy array
print(cm.tp, cm.fn, cm.fp, cm.tn)               # Get the individual quadrant values

print(cm.get_latex_table(multirow=True))        # Get the matrix as code for a Latex table

print(cm.recall_score(average='micro'))         # Call any sklearn.metrics function using the data in the matrix

ROCCurve

from tremetrics import ROCCurve
import matplotlib.pyplot as plt

y_true, y_pred_prob = ...                       # Generate prediction probabilities

curve = ROCCurve.from_pred_prob(y_true, y_pred_prob)  # Create a new roc curve object
print(curve.roc_auc)                            # Print the area under the roc curve

curve.plot()                                    # plot the roc curve
plt.show()                                      # show the curve plot

# generating convex roc curves
curve = ROCCurve.from_pred_prob(y_true, y_pred_prob, convex=True)  # Create a convex roc curve object
print(curve.roc_auc)                            # Print the area under the roc curve

curve.plot()                                    # plot the roc curve
plt.show()                                      # show the curve plot

# static methods meant as sklearn drop-in replacement
fpr, tpr, thresholds = ROCCurve.roc_curve(y_true, y_pred_prob)
fpr, tpr, thresholds = ROCCurve.convex_roc_curve(y_true, y_pred_prob)

# exploiting pyplot state to draw both standard and convex roc curves
curve = ROCCurve.from_pred_prob(y_true, y_pred_prob)
curve.plot()
curve = ROCCurve.from_pred_prob(y_true, y_pred_prob, convex=True)
curve.plot()
plt.show()

Why should I make my ROC curve convex?

A nice explanation is given in this paper.

In essence, ROC points below the convex hull are suboptimal classifiers, which can be outperformed by interpolating nearby classifiers that are on the convex hull. Practically, this interpolation is a random choice (i.e. without considering input data or predictions made) between such better classifiers, where the probability of picking one over the other determines the position on the line segment between their ROC points.