shap-adaptive-boosting 1.0.0

Versions of AdaBoost and RUSBoost compatible with TreeSHAP and respective TreeSHAP Explainers.

SHAP for Adaptive Boosting

TreeSHAP-compatible AdaBoost and RUSBoost classifiers for explainable AI on imbalanced datasets.

Overview

This package provides custom implementations of AdaBoost and RUSBoost classifiers that are compatible with SHAP's TreeExplainer, enabling fast local explanations for boosting algorithms on imbalanced datasets. The implementation is based on research from my master's thesis "A Novel Approach to Explainability in Tree-Based Ensemble Algorithms for Imbalanced Datasets" (Leipzig University and Alexander Thamm GmbH, 2023).

Key Features

• TreeSHAP Compatibility: Modified boosting algorithms that work with SHAP's TreeExplainer
• Imbalanced Dataset Support: RUSBoost implementation for handling class imbalance
• Fast Explanations: Polynomial time complexity for SHAP value computation
• Custom Explainers: Dedicated explainer classes for both algorithms

📁 Project Structure

.
├── src/
│   └── shap_adaptive_boosting/
│       ├── classifiers.py    # Custom AdaBoost and RUSBoost implementations
│       └── explainers.py     # TreeSHAP-compatible explainers
└── tests/
    ├── test_adaboost.py
    ├── test_adaboost_explainer.py
    ├── test_rusboost.py
    └── test_rusboost_explainer.py

Installation

This project uses Poetry for dependency management.

poetry install

pre-commit autoupdate
pre-commit install

Usage

Basic Example

from shap import Explanation
from shap.datasets import adult
from shap.plots import beeswarm
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

from shap_adaptive_boosting.classifiers import RUSBoostClassifier
from shap_adaptive_boosting.explainers import RUSBoostExplainer

X_train, X_test, y_train, y_test = train_test_split(
    *adult(), test_size=0.2, random_state=42
)

rbc = RUSBoostClassifier(
    estimator=DecisionTreeClassifier(max_depth=10),
    learning_rate=0.1,
    n_estimators=50,
    random_state=0,
    sampling_strategy=1 / 3,
    replacement=True,
)
rbc.fit(X_train, y_train)

predictions = rbc.predict_proba(X_test)

rbe = RUSBoostExplainer(rbc, X_train)
shap_values = rbe.shap_values(X_test)

beeswarm(
    shap_values=Explanation(
        values=shap_values[:, :, 0],
        base_values=rbe.expected_value[0],
        data=X_test,
        feature_names=adult()[0].columns,
    )
)

Custom Classifiers

• AdaBoostClassifier: Modified SAMME algorithm compatible with TreeSHAP
• RUSBoostClassifier: Combines AdaBoost with random undersampling for imbalanced datasets

Custom Explainers

• AdaBoostExplainer: Generates SHAP explanations for Custom AdaBoost
• RUSBoostExplainer: Generates SHAP explanations for Custom RUSBoost

Testing

Run the test suite:

poetry run pytest

Technical Details

TreeSHAP Integration

The implementation modifies the predict_proba method of AdaBoost and RUSBoost to use linear probability calculations instead of log-transformed probabilities, ensuring compatibility with TreeSHAP's framework.

SHAP Value Computation

Both explainers calculate SHAP values by:

1. Creating individual TreeExplainer instances for each weak learner
2. Computing weighted averages of SHAP values across the ensemble
3. Maintaining compatibility with SHAP's visualization tools