Forest-Guided Clustering - Shedding light into the Random Forest Black Box

✨ About this Package

Why Use Forest-Guided Clustering?

Forest-Guided Clustering (FGC) is an explainability method for Random Forest models that addresses one of the key limitations of many standard XAI techniques: the inability to effectively handle correlated features and complex decision patterns. Traditional methods like permutation importance, SHAP, and LIME often assume feature independence and focus on individual feature contributions, which can lead to misleading or incomplete explanations. As machine learning models are increasingly deployed in sensitive domains like healthcare, finance, and HR, understanding why a model makes a decision is as important as the decision itself. This is not only a matter of trust and fairness, but also a legal requirement in many jurisdictions, such as the European Union's GDPR which mandates a “right to explanation” for automated decisions.

FGC offers a different approach: instead of approximating the model with simpler surrogates, it uses the internal structure of the Random Forest itself. By analyzing the tree traversal patterns of individual samples, FGC clusters data points that follow similar decision paths. This reveals how the forest segments the input space, enabling a human-interpretable view of the model's internal logic. FGC is particularly useful when features are highly correlated, as it does not rely on assumptions of feature independence. It bridges the gap between model accuracy and model transparency, offering a powerful tool for global, model-specific interpretation of Random Forests.

📢 New! Forest-Guided Clustering is now on arXiv

Please see our paper Forest-Guided Clustering - Shedding Light into the Random Forest Black Box for a detailed description of the method, its theoretical foundations, and practical applications. Check it out to learn more about how FGC reveals structure in your Random Forest models!

Prefer a visual walkthrough? Watch our short introduction video by clicking below:

Curious how Forest-Guided Clustering compares to standard methods? See our notebook: Introduction to FGC: Comparison of Forest-Guided Clustering and Feature Importance.

Want to dive deeper? Visit our full documentation for:

• Getting Started – Installation and quick start
• Tutorials – Use cases for classification, regression, and large datasets
• API Reference – Detailed descriptions of functions and classes

🛠️ Installation

Requirements

This package was tested for Python 3.10 - 3.13 on ubuntu, macos and windows. It depends on the kmedoids python package. If you are using windows or macos, you may need to first install Rust/Cargo with:

conda install -c conda-forge rust

If this does not work, please try to install Cargo from source:

git clone https://github.com/rust-lang/cargo
cd cargo
cargo build --release

For further information on the kmedoids package, please visit this page.

All other required packages are automatically installed if installation is done via pip.

Install Options

The installation of the package is done via pip. Note: if you are using conda, first install pip with: conda install pip.

PyPI install:

pip install fgclustering

Installation from source:

git clone https://github.com/HelmholtzAI-Consultants-Munich/fg-clustering.git

• Installation as python package (run inside directory):

    pip install .   
  

💻 How to Use Forest-Guided Clustering

Basic Usage

To apply Forest-Guided Clustering (FGC) for explaining a Random Forest model, you can follow the simple workflow consisting of three main steps: computing the forest-guided clusters, evaluating feature importance, and visualizing the results.

# for classification tasks
clustering_distance_metric = DistanceRandomForestProximity()
# for regression tasks
clustering_distance_metric = DistanceRandomForestLCA()

# compute the forest-guided clusters
fgc = forest_guided_clustering(
    estimator=model, 
    X=X, 
    y=y, 
    clustering_distance_metric=clustering_distance_metric, 
    clustering_strategy=ClusteringKMedoids(),
)

# evaluate feature importance for best k
fgc_fi = forest_guided_feature_importance(
    X=X, 
    y=y, 
    y_pred=model.predict(X),
    cluster_labels=fgc.cluster_labels[fgc.best_k],
)

# visualize the results
plot_forest_guided_clustering(
    ks=fgc.ks, 
    scores=fgc.scores, 
    mean_ji=fgc.mean_ji, 
    cluster_jis=fgc.cluster_jis, 
    best_k=fgc.best_k, 
)

plot_forest_guided_feature_importance(
    feature_importance_local=fgc_fi.feature_importance_local,
    feature_importance_global=fgc_fi.feature_importance_global,
)

plot_forest_guided_decision_paths(
    data_clustering=fgc_fi.data_clustering,
    feature_importance_global=fgc_fi.feature_importance_global,
    feature_importance_local=fgc_fi.feature_importance_local,
    model_type=fgc.model_type,
)

where

• estimator is the trained Random Forest model
• X is the feature matrix
• y is the target variable
• clustering_distance_metric defines how similarity between samples is measured based on the Random Forest structure
• clustering_strategy determines how the distance-based clustering is performed

For a detailed walkthrough, refer to the Introduction to FGC: Simple Use Cases notebook.

Using FGC on Large Datasets

When working with datasets containing a large number of samples, Forest-Guided Clustering (FGC) provides several strategies to ensure efficient performance and scalability:

• Parallelize Cluster Optimization: Leverage multiple CPU cores by setting the n_jobs parameter to a value greater than 1 in the forest_guided_clustering() function. This will parallelize the bootstrapping process for evaluating cluster stability.

• Use a Faster Clustering Algorithm: Improve the efficiency of the K-Medoids clustering step by using the optimized "fasterpam" algorithm. Set the method parameter of your clustering strategy (e.g., ClusteringKMedoids(method="fasterpam")) to activate this faster implementation.

• Enable Subsampling with CLARA: For extremely large datasets, consider using the CLARA (Clustering Large Applications) variant by choosing ClusteringClara() as your clustering strategy. CLARA performs clustering on smaller random subsamples, making it suitable for high-volume data.

For a detailed example, please refer to the notebook Special Case: FGC for Big Datasets.

🤝 Contributing

We welcome contributions of all kinds—whether it’s improvements to the code, documentation, tutorials, or examples. Your input helps make Forest-Guided Clustering more robust and useful for the community.

To contribute:

1. Fork the repository.
2. Make your changes in a feature branch.
3. Submit a pull request to the main branch.

We’ll review your submission and work with you to get it merged.

If you have any questions or ideas you'd like to discuss before contributing, feel free to reach out to Lisa Barros de Andrade e Sousa.

📝 How to cite

If you find Forest-Guided Clustering useful in your research or applications, please consider citing it:

@article{barros2025forest,
    title	= {Forest-Guided Clustering -- Shedding Light into the Random Forest Black Box},
    author	= {Lisa Barros de Andrade e Sousa,
		   Gregor Miller,
		   Ronan Le Gleut,
		   Dominik Thalmeier,
		   Helena Pelin,
		   Marie Piraud},
    journal	= {ArXiv},
    year	= {2025},
    url         = {https://doi.org/10.48550/arXiv.2507.19455}
}

🛡️ License

The fgclustering package is released under the MIT License. You are free to use, modify, and distribute it under the terms outlined in the LICENSE file.