intrinsic-kappa 0.1.1

Prevalence-independent intrinsic kappa coefficient for classification systems with any number of categories.

intrinsic-kappa

A Python library for computing the prevalence-independent intrinsic kappa coefficient and its one-sided confidence lower bounds for classification systems with any number of categories (NC ≥ 2).

Background

Traditional Cohen's kappa depends on class prevalence — the proportion of instances in each category — which can produce misleading performance estimates when a dataset is imbalanced. The intrinsic kappa solves this by anchoring the chance-agreement term exclusively to the number of categories (1/NC), providing a robust and stable measure of a classifier's inherent discriminatory ability, regardless of sample composition.

This package is based on the methodology published in:

Sanchez-Marquez, R. et al. (2025). A statistical approach to the confusion matrix for classification problems using machine learning. Computers & Industrial Engineering.

Installation

pip install intrinsic-kappa