FIPE: Functionally Identical Pruning of Ensembles
This repository provides methods for Functionally-Identical Pruning of Tree Ensembles (FIPE). Given a trained scikit-learn model, FIPE provides a pruned model that is certified to be equivalent to the original model on the entire feature space. The algorithm is described in detail in the paper: https://arxiv.org/abs/2408.16167 .
Installation
This project requires the gurobi solver. Free academic licenses are available. Please consult:
Run the following commands from the project root to install the requirements. You may have to install python and venv before.
virtualenv -p python3.12 env
pip install fipepy
The installation can be checked by running the test suite:
pip install tox
tox
The integration tests require a working Gurobi license. If a license is not available, the tests will pass and print a warning.
Getting started
A minimal working example to prune an AdaBoost ensemble is presented below.
import gurobipy as gp
import numpy as np
import pandas as pd
from sklearn.datasets import load_iris
from sklearn.ensemble import AdaBoostClassifier
from sklearn.model_selection import train_test_split
from fipe import FIPE, FeatureEncoder
# Load data encode features
data = load_iris(as_frame=True)
X = pd.DataFrame(data.data)
y = data.target
encoder = FeatureEncoder(X)
X = encoder.X.to_numpy()
# Train tree ensemble
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
base = AdaBoostClassifier(n_estimators=100, random_state=42)
base.fit(X, y)
# Read and normalize weights
w = base.estimator_weights_
w = (w / w.max()) * 1e5
# Prune using FIPE
norm = 1
print(f"Pruning model by minimizing l_{norm} norm.")
env = gp.Env()
env.setParam("OutputFlag", 0)
pruner = FIPE(
base=base,
encoder=encoder,
weights=w,
norm=norm,
env=env,
eps=1e-6,
tol=1e-4,
)
print("Building pruner...")
pruner.build()
pruner.add_samples(X_train)
print("Pruning...")
pruner.prune()
print("Finished pruning.")
# Read pruned model
n_active_estimators = pruner.n_active_estimators
print(
f"The pruned ensemble has {n_active_estimators}"
f"/{base.n_estimators} active estimators."
)
# Verify functionally-identical on test data
y_pred = base.predict(X_test)
y_pruned = pruner.predict(X_test)
fidelity = np.mean(y_pred == y_pruned)
print(f"Fidelity to initial ensemble is {fidelity * 100:.2f}%.")
Metadata
Release files for fipepy 1.0.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| fipepy-1.0.7.tar.gz | 218.2 kB | Without provenance | Details for fipepy-1.0.7.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| fipepy-1.0.7-py3-none-any.whl | Python 3 | none | any | Details for fipepy-1.0.7-py3-none-any.whl |
Total release size: 252.7 kB
Release files / fipepy-1.0.7.tar.gz
| Download URL | fipepy-1.0.7.tar.gz |
|---|---|
| Size | 218.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c8497d1d3b292500f309abc8002c96cfe8a54255704cd8b956e9153eabb57160
|
|
BLAKE2b-256 checksum How to use checksums |
4dcb0eeea4ba956756c2e1e4822d00786f7ad17d710810df8cdb0aa47e9ba9fe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.0.1 CPython/3.9.21
|
Release files / fipepy-1.0.7-py3-none-any.whl
| Download URL | fipepy-1.0.7-py3-none-any.whl |
|---|---|
| Size | 34.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
3231ec38bba1afe1004d988c28d8f56bfe7913979f66eed0c0a31b5b2ec48e86
|
|
BLAKE2b-256 checksum How to use checksums |
2151b86eab3c0347ba7b1c6a3f857f37d186e37a2ff2888271e65b1215fb8aa5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.0.1 CPython/3.9.21
|
Security for fipepy 1.0.7
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.