Algorithms for monitoring and explaining machine learning models
Project description
Supervisor is a Python library aimed at machine learning model inspection and interpretation. The focus of the library is to provide high-quality implementations of black-box, white-box, local and global explanation methods for classification and regression models.
If you're interested in outlier detection, concept drift or adversarial instance detection, check out our sister project supervisor-detect.
Anchor explanations for images |
Integrated Gradients for text |
Counterfactual examples |
Accumulated Local Effects |
Table of Contents
Installation and Usage
Supervisor can be installed from:
- PyPI or GitHub source (with
pip
) - Anaconda (with
conda
/mamba
)
With pip
-
Supervisor can be installed from PyPI:
pip install supervisor
-
Alternatively, the development version can be installed:
pip install git+https://github.com/kozmoai/supervisor.git
-
To take advantage of distributed computation of explanations, install
supervisor
withray
:pip install supervisor[ray]
-
For SHAP support, install
supervisor
as follows:pip install supervisor[shap]
With conda
To install from conda-forge it is recommended to use mamba, which can be installed to the base conda enviroment with:
conda install mamba -n base -c conda-forge
-
For the standard Supervisor install:
mamba install -c conda-forge supervisor
-
For distributed computing support:
mamba install -c conda-forge supervisor ray
-
For SHAP support:
mamba install -c conda-forge supervisor shap
Usage
The supervisor explanation API takes inspiration from scikit-learn
, consisting of distinct initialize,
fit and explain steps. We will use the AnchorTabular
explainer to illustrate the API:
from supervisor.explainers import AnchorTabular
# initialize and fit explainer by passing a prediction function and any other required arguments
explainer = AnchorTabular(predict_fn, feature_names=feature_names, category_map=category_map)
explainer.fit(X_train)
# explain an instance
explanation = explainer.explain(x)
The explanation returned is an Explanation
object with attributes meta
and data
. meta
is a dictionary
containing the explainer metadata and any hyperparameters and data
is a dictionary containing everything
related to the computed explanation. For example, for the Anchor algorithm the explanation can be accessed
via explanation.data['anchor']
(or explanation.anchor
). The exact details of available fields varies
from method to method so we encourage the reader to become familiar with the
types of methods supported.
Supported Methods
The following tables summarize the possible use cases for each method.
Model Explanations
Method | Models | Explanations | Classification | Regression | Tabular | Text | Images | Categorical features | Train set required | Distributed |
---|---|---|---|---|---|---|---|---|---|---|
ALE | BB | global | ✔ | ✔ | ✔ | |||||
Partial Dependence | BB WB | global | ✔ | ✔ | ✔ | ✔ | ||||
PD Variance | BB WB | global | ✔ | ✔ | ✔ | ✔ | ||||
Permutation Importance | BB | global | ✔ | ✔ | ✔ | ✔ | ||||
Anchors | BB | local | ✔ | ✔ | ✔ | ✔ | ✔ | For Tabular | ||
CEM | BB* TF/Keras | local | ✔ | ✔ | ✔ | Optional | ||||
Counterfactuals | BB* TF/Keras | local | ✔ | ✔ | ✔ | No | ||||
Prototype Counterfactuals | BB* TF/Keras | local | ✔ | ✔ | ✔ | ✔ | Optional | |||
Counterfactuals with RL | BB | local | ✔ | ✔ | ✔ | ✔ | ✔ | |||
Integrated Gradients | TF/Keras | local | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | Optional | |
Kernel SHAP | BB | local global |
✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ||
Tree SHAP | WB | local global |
✔ | ✔ | ✔ | ✔ | Optional | |||
Similarity explanations | WB | local | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
Model Confidence
These algorithms provide instance-specific scores measuring the model confidence for making a particular prediction.
Method | Models | Classification | Regression | Tabular | Text | Images | Categorical Features | Train set required |
---|---|---|---|---|---|---|---|---|
Trust Scores | BB | ✔ | ✔ | ✔(1) | ✔(2) | Yes | ||
Linearity Measure | BB | ✔ | ✔ | ✔ | ✔ | Optional |
Key:
- BB - black-box (only require a prediction function)
- BB* - black-box but assume model is differentiable
- WB - requires white-box model access. There may be limitations on models supported
- TF/Keras - TensorFlow models via the Keras API
- Local - instance specific explanation, why was this prediction made?
- Global - explains the model with respect to a set of instances
- (1) - depending on model
- (2) - may require dimensionality reduction
Prototypes
These algorithms provide a distilled view of the dataset and help construct a 1-KNN interpretable classifier.
Method | Classification | Regression | Tabular | Text | Images | Categorical Features | Train set labels |
---|---|---|---|---|---|---|---|
ProtoSelect | ✔ | ✔ | ✔ | ✔ | ✔ | Optional |
References and Examples
-
Accumulated Local Effects (ALE, Apley and Zhu, 2016)
- Documentation
- Examples: California housing dataset, Iris dataset
-
Partial Dependence (J.H. Friedman, 2001)
- Documentation
- Examples: Bike rental
-
Partial Dependence Variance(Greenwell et al., 2018)
- Documentation
- Examples: Friedman’s regression problem
-
Permutation Importance(Breiman, 2001; Fisher et al., 2018)
- Documentation
- Examples: Who's Going to Leave Next?
-
Anchor explanations (Ribeiro et al., 2018)
-
Contrastive Explanation Method (CEM, Dhurandhar et al., 2018)
- Documentation
- Examples: MNIST, Iris dataset
-
Counterfactual Explanations (extension of Wachter et al., 2017)
- Documentation
- Examples: MNIST
-
Counterfactual Explanations Guided by Prototypes (Van Looveren and Klaise, 2019)
-
Model-agnostic Counterfactual Explanations via RL(Samoilescu et al., 2021)
- Documentation
- Examples: MNIST, Adult income
-
Integrated Gradients (Sundararajan et al., 2017)
- Documentation,
- Examples: MNIST example, Imagenet example, IMDB example.
-
Kernel Shapley Additive Explanations (Lundberg et al., 2017)
-
Tree Shapley Additive Explanations (Lundberg et al., 2020)
-
Trust Scores (Jiang et al., 2018)
- Documentation
- Examples: MNIST, Iris dataset
-
Linearity Measure
- Documentation
- Examples: Iris dataset, fashion MNIST
-
ProtoSelect
- Documentation
- Examples: Adult Census & CIFAR10
-
Similarity explanations
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for kozmo_supervisor-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2929c3fcce9051eab260620411d7eaa3685a337d6fd192f472a6e694aff955c0 |
|
MD5 | f0ea605896077a7c0e0eaa75e1ee342b |
|
BLAKE2b-256 | e2151e7338bc89846d1d635f6a60ea41e19babd45c5d40d3f754e0b9b94a20b6 |