CGA2M_plus is a Python package of Constraint GA2M plus(CGA2M+). CGA2M+ is a model that improves GA2M in two respects: interpretability and accuracy.
Project description
CGA2M+ (Constraint GA2M plus)
We propose Constraint GA2M plus (CGA2M+), which we proposed. CGA2M+ is a modified version of GA2M to improve its interpretability and accuracy. For more information, please read our paper.(coming soon!!) Mainly, CGA2M+ differs from GA2M in two respects.
- introducing monotonic constraints
- introducing higher-order interactions keeping the interpretability of the model
Description of CGA2M+
Mainly, CGA2M+ differs from GA2M in two respects. We are using LightGBM as a shape function.
- introducing monotonic constraints
By adding monotonicity, we can improve the interpretability of our model. For example, we can make sure that "in the real estate market, as the number of rooms increases, the price decreases" does not happen. Human knowledge is needed to determine which features to enforce monotonicity on. The monotonicity constraint algorithm is implemented in LightGBM. This is a way to constrain the branches of a tree. For more details, please refer to the LightGBM implementation.
- introducing higher-order interactions keeping the interpretability of the model
GGA2M is unable to take into account higher-order interactions. Therefore, we introduce higher-order terms that are not interpretable. However, we devise a learning method so that the higher-order terms do not compromise the overall interpretability. Specifically, we train the higher-order terms as models that predict the residuals of the univariate terms and pairwise interaction terms. This allows most of the predictions to be explained by the interpretable first and second order terms. These residuals are then predicted by a higher-order term.
Algorithm
For more information, please read our paper. (coming soon!!)
Installation
You can get CGA2M+ from PyPI. Our project in PyPI is here.
pip install cga2m-plus
Usage
For more detail, please read examples/How_to_use_CGA2M+.ipynb
.
If it doesn't render at all in github, please click here.
Training
cga2m = Constraint_GA2M(X_train,
y_train,
X_eval,
y_eval,
lgbm_params,
monotone_constraints = [0] * 6,
all_interaction_features = list(itertools.combinations(range(X_test.shape[1]), 2)))
cga2m.train(max_outer_iteration=20,backfitting_iteration=20,threshold=0.05)
cga2m.prune_and_retrain(threshold=0.05,backfitting_iteration=30)
cga2m.higher_order_train()
Predict
cga2m.predict(X_test,higher_mode=True)
Visualize the effect of features on the target variables.
plot_main(cga2m_no1,X_train)
Visualize (3d) the effect of pairs of features on the target variables
plot_interaction(cga2m_no1,X_train,mode = "3d")
Feature importance
show_importance(cga2m_no1,after_prune=True,higher_mode=True)
License
MIT License
Citation
You may use our package(CGA2M+) under MIT License. If you use this program in your research then please cite:
CGA2M+ Package
@misc{kuramata2021cga2mplus,
author = {Michiya, Kuramata and Akihisa, Watanabe and Kaito, Majima and Kensyo, Kondo and Kazuhide, Nakata},
title = {Constraint GA2M plus},
year = {2021},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/MK-tech20/CGA2M_plus}}
}
CGA2M+ Paper
The paper has not been published yet.
Reference
[1] Friedman, J. H. 2001, Greedy function approximation: a gradient boosting machine, Annals of statistics, 1189-1232, doi: 10.1214/aos/1013203451. Available online: May 02, 2021
[2] Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., ... Liu, T. Y. 2017. Lightgbm: A highly efficient gradient boosting decision tree, Advances in neural information processing systems(NIPS’17), Long Beach California , 4-9 December, pp. 3146-3154.
[3] Nelder, J. A., Wedderburn, R. W. 1972. Generalized linear models, Journal of the Royal Statistical Society: Series A (General), 135(3), 370-384, doi: 10.2307/2344614, Available online: May 02, 2021
[4] Hastie, T. J., Tibshirani, R. J. 1990. Generalized additive models (Vol. 43), CRC press, doi: 10.1214/ss/1177013604. Available online: May 02, 2021
[5] Lou, Y., Caruana, R., Gehrke, J., Hooker, G. 2013, August. Accurate intelligible models with pairwise interactions, Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining(KDD’13), Chicago Illinois, United States of America, 11-14 August, pp. 623-631.
[6] “GitHub - microsoft/LightGBM” [Online]. Available: https://github.com/microsoft/LightGBM (Accessed: May 02, 2021)
[7] “scikit-learn: machine learning in Python — scikit-learn 0.24.2 documentation” [Online]. Available: https://scikit-learn.org/stable/ (Accessed May 02, 2021)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for CGA2M_plus-0.1.7-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4a8f80ecf229bc3186a8d724cf91a1dfde4c49ad62a88199d0edbd21a74447fd |
|
MD5 | 02bd1a73549bc75e5cf0dc4eb0befbde |
|
BLAKE2b-256 | f70547598022392c93e8a794109ccfd422b3c736b9f5f243f98b15d538975d5a |