Using HBAC to detect biased data segments
- Hierarchical Bias-Aware Clustering (HBAC) on regression models.
- Input: a trained model and a model's test data.
- Output: analysis of biased/discriminated data segment according to HBAC:
- Comparing distributions of discriminated and remaining data.
- Segment predictor: trains a XGBoost binary classifier to evaluate distinguishability of discriminated and remaining data with descriptive features.
# Initialize HBAC
hbac = HBAC_analyser()
# In this case, input includes model path, X data and Y data
hbac.hbac_on_model(model_path, X_test, y_test)
hbac.pca_plot()
discrimated_cluster, bias = hbac.get_max_bias_cluster(print_results=True)
# Get discriminated data in panda df
hbac.all_unscaled_discriminated
# Displaying results in dataframes
hbac.clustered_data
# Mean per feature 'discrimnated' cluster vs 'remaining' clusters
hbac.mean_clusters
# Plot 3 most different features' distributions
hbac.plot_distributions(plot_top_features = 3)
# Train XGBoost a binary classifier to predict whther a datapoint will be discrimnated or not, without using error as feature.
hbac.segment_predictor(plot_roc_auc=True,shap_analysis=True)
Also see example.ipynb.
For the use of HBAC on classification models, see https://github.com/Sm2468/msc_thesis/tree/master/hbac%20scripts, on which this project was based.
1 maintainer
Metadata
1 maintainer
Release files for hbac-bias-detection 0.2.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| hbac_bias_detection-0.2.6-py3-none-any.whl | Python 3 | none | any | Details |
| hbac_bias_detection-0.2.6-py2.py3-none-any.whl | Python 2, Python 3 | none | any | Details |
Total release size: 25.9 kB
Release files / hbac_bias_detection-0.2.6-py3-none-any.whl
| Download URL | hbac_bias_detection-0.2.6-py3-none-any.whl |
|---|---|
| Size | 13.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
abbfeba34c5ccd7f27052b74dcbd4ed960e05dddd1d51b9fe33fd7a22725aa13
|
|
BLAKE2b-256 checksum How to use checksums |
4d1e86ab95d14a1383bfd32f7c5f2f526d672bdc61db1cca634093359837adc8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.0 CPython/3.9.2
|
Release files / hbac_bias_detection-0.2.6-py2.py3-none-any.whl
| Download URL | hbac_bias_detection-0.2.6-py2.py3-none-any.whl |
|---|---|
| Size | 13.0 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
98cfa6155864ddcdaf88dabb15d35413e5f18d06cdd3caf51dfb4026b34c6278
|
|
BLAKE2b-256 checksum How to use checksums |
a81eb0f4f21a2334d86fa5efc640c42cb79674dfeee8a4dc7154d300528362c3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.0 CPython/3.9.2
|
