HyperSHAP

HyperSHAP – a game‑theoretic Python library for explaining Hyperparameter Optimization (HPO). It uses Shapley values and interaction indices to provide both local and global insights into how individual hyper‑parameters (and their interactions) affect a model’s performance.

• Github repository: https://github.com/automl/HyperSHAP/
• Documentation https://automl.github.io/HyperSHAP/

Table of Contents

• Features
• Installation
• Getting Started
• API Overview
• Example Notebook
• Citation
• Contributing
• License

Features

• Additive Shapley decomposition of any performance metric across hyper‑parameters.
• Interaction analysis via the Faithful Shapley Interaction Index (FSII).
• Ready‑made explanation tasks for Ablation, Tunability, and Optimizer Bias studies.
• Integrated visualisation (SI‑graph) for interaction effects.
• Works with any surrogate model that follows the ExplanationTask interface.

Installation

First, create a virtual environment, e.g., via conda:

$ conda create -n hypershap python=3.10
$ conda activate hypershap

Now, you can just pip install HyperSHAP as follows:

$ pip install hypershap

Or, clone the git repository and install hypershap via the Makefile:

$ git clone https://github.com/automl/hypershap
$ cd hypershap
$ make install

Getting Started

Given an existing setup with a ConfigurationSpace from the ConfigSpace package and black-box function as follows:

from ConfigSpace import ConfigurationSpace, Configuration

# ConfigurationSpace describing the hyperparameter space
cs = ConfigurationSpace()
  ...

# A black-box function, evaluating ConfigSpace.Configuration objects
def blackbox_function(cfg: Configuration) -> float:
  ...

You can use HyperSHAP as follows:

from hypershap import ExplanationTask, HyperSHAP

# Instantiate HyperSHAP
hypershap = HyperSHAP(ExplanationTask.from_function(config_space=cs,function=blackbox_function))
# Conduct tunability analysis
hypershap.tunability(baseline_config=cs.get_default_configuration())
# Plot results as a Shapley Interaction graph
hypershap.plot_si_graph()

The example demonstrates how to:

1. Wrap a black-box function in an explanation task.
2. Use HyperSHAP to obtain interaction values for the tunability game.
3. Plot the corresponding SI-graph.

API Overview

All methods return an InteractionValues object (from shapiq) that can be inspected, saved, or passed to the visualisation routine.

Example Notebooks

Full Jupyter notebooks illustrating all three explanation tasks (ablation, tunability, optimizer bias) are included in the repository under examples/. The notebookts walk through:

• Building a mockup environment
• Creating the corresponding explanation task
• Loading explanation tasks from different setups: data, black-box function, and existing surrogate model.
• Computing interaction values with HyperSHAP
• Visualizing results with plot_si_graph

Citation

If you use HyperSHAP in your research, please cite the original paper:

@article{wever-arxiv25,
  author       = {Marcel Wever and
                  Maximilian Muschalik and
                  Fabian Fumagalli and
                  Marius Lindauer},
  title        = {HyperSHAP: Shapley Values and Interactions for Hyperparameter Importance},
  journal      = {CoRR},
  volume       = {abs/2502.01276},
  year         = {2025},
  doi          = {10.48550/ARXIV.2502.01276},
}

The paper introduces the underlying game-theoretic framework and demonstrates its usefulness for HPO explainability.

Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repo and create a feature branch (git checkout -b feat/your-feature).
2. Write tests (the project uses pytest).
3. Ensure all tests pass (pytest).
4. Update documentation if you add new functionality.
5. Submit a Pull Request with a clear description of the changes.

See CONTRIBUTING.md for detailed guidelines.

License

HyperSHAP is released under the BSD 3-Clause License. See the LICENSE file for full terms.

Enjoy exploring your HPO pipelines with HyperSHAP! 🎉

Repository initiated with fpgmaas/cookiecutter-uv.