A helper library to jumpstart your machine learning project based on tabular or structured data.
Project description
Getting Started Tutorial with TMLT (Tabular ML Toolkit)
A tutorial on getting started with TMLT (Tabular ML Toolkit)
%load_ext autoreload
%autoreload 2
Install
pip install -U tabular_ml_toolkit
How to Best Use tabular_ml_toolkit
Start with your favorite model and then just simply create MLPipeline with one API.
For example, Here we are using RandomForestRegressor from Scikit-Learn, on Melbourne Home Sale price data
No need to install scikit-learn as it comes preinstall with Tabular_ML_Toolkit
from tabular_ml_toolkit.tmlt import *
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_absolute_error
import numpy as np
# for displaying diagram of pipelines
from sklearn import set_config
set_config(display="diagram")
# Just to compare fit times
import time
# Dataset file names and Paths
DIRECTORY_PATH = "input/home_data/"
TRAIN_FILE = "train.csv"
TEST_FILE = "test.csv"
SAMPLE_SUB_FILE = "sample_submission.csv"
OUTPUT_PATH = "output/"
from xgboost import XGBRegressor
xgb_params = {
'use_label_encoder':False,
'eval_metric':'rmse',
'random_state':42,
# for GPU
# 'tree_method': 'gpu_hist',
# 'predictor': 'gpu_predictor',
}
# create xgb ml model
xgb_model = XGBRegressor(**xgb_params)
Just point in the direction of your data, let tmlt know what are idx and target columns in your tabular data and what kind of problem type you are trying to resolve
# tmlt
tmlt = TMLT().prepare_data_for_training(
train_file_path= DIRECTORY_PATH+TRAIN_FILE,
test_file_path= DIRECTORY_PATH+TEST_FILE,
idx_col="Id", target="SalePrice",
model=xgb_model,
random_state=42,
problem_type="regression")
2021-11-22 16:57:13,379 INFO 12 cores found, model and data parallel processing should worked!
2021-11-22 16:57:13,432 INFO DataFrame Memory usage decreased to 0.58 Mb (35.5% reduction)
2021-11-22 16:57:13,479 INFO DataFrame Memory usage decreased to 0.58 Mb (34.8% reduction)
2021-11-22 16:57:13,512 INFO Both Numerical & Categorical columns found, Preprocessing will done accordingly!
# check sklearn pipeline
tmlt.spl
Pipeline
Pipeline(steps=[('preprocessor',
ColumnTransformer(transformers=[('num_cols',
Pipeline(steps=[('imputer',
SimpleImputer(strategy='constant')),
('scaler',
StandardScaler())]),
['MSSubClass', 'LotFrontage',
'LotArea', 'OverallQual',
'OverallCond', 'YearBuilt',
'YearRemodAdd', 'MasVnrArea',
'BsmtFinSF1', 'BsmtFinSF2',
'BsmtUnfSF', 'TotalBsmtSF',
'1stFlrSF', '2ndFlrSF',
'...
interaction_constraints=None, learning_rate=None,
max_delta_step=None, max_depth=None,
min_child_weight=None, missing=nan,
monotone_constraints=None, n_estimators=100,
n_jobs=11, num_parallel_tree=None, predictor=None,
random_state=42, reg_alpha=None, reg_lambda=None,
scale_pos_weight=None, subsample=None,
tree_method=None, use_label_encoder=False,
validate_parameters=None, verbosity=None))])</pre></div></div></div><div class="sk-serial"><div class="sk-item sk-dashed-wrapped"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="d3de4fb1-75e0-4e84-87bf-e1e7dee35388" type="checkbox" ><label class="sk-toggleable__label" for="d3de4fb1-75e0-4e84-87bf-e1e7dee35388">preprocessor: ColumnTransformer</label><div class="sk-toggleable__content"><pre>ColumnTransformer(transformers=[('num_cols',
Pipeline(steps=[('imputer',
SimpleImputer(strategy='constant')),
('scaler', StandardScaler())]),
['MSSubClass', 'LotFrontage', 'LotArea',
'OverallQual', 'OverallCond', 'YearBuilt',
'YearRemodAdd', 'MasVnrArea', 'BsmtFinSF1',
'BsmtFinSF2', 'BsmtUnfSF', 'TotalBsmtSF',
'1stFlrSF', '2ndFlrSF', 'LowQualFinSF',
'GrLivArea', 'BsmtF...
['MSZoning', 'Street', 'Alley', 'LotShape',
'LandContour', 'Utilities', 'LotConfig',
'LandSlope', 'Condition1', 'Condition2',
'BldgType', 'HouseStyle', 'RoofStyle',
'RoofMatl', 'MasVnrType', 'ExterQual',
'ExterCond', 'Foundation', 'BsmtQual',
'BsmtCond', 'BsmtExposure', 'BsmtFinType1',
'BsmtFinType2', 'Heating', 'HeatingQC',
'CentralAir', 'Electrical', 'KitchenQual',
'Functional', 'FireplaceQu', ...])])</pre></div></div></div><div class="sk-parallel"><div class="sk-parallel-item"><div class="sk-item"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="08f50cb5-2f79-4b79-ac8c-8a2db42f7fbb" type="checkbox" ><label class="sk-toggleable__label" for="08f50cb5-2f79-4b79-ac8c-8a2db42f7fbb">num_cols</label><div class="sk-toggleable__content"><pre>['MSSubClass', 'LotFrontage', 'LotArea', 'OverallQual', 'OverallCond', 'YearBuilt', 'YearRemodAdd', 'MasVnrArea', 'BsmtFinSF1', 'BsmtFinSF2', 'BsmtUnfSF', 'TotalBsmtSF', '1stFlrSF', '2ndFlrSF', 'LowQualFinSF', 'GrLivArea', 'BsmtFullBath', 'BsmtHalfBath', 'FullBath', 'HalfBath', 'BedroomAbvGr', 'KitchenAbvGr', 'TotRmsAbvGrd', 'Fireplaces', 'GarageYrBlt', 'GarageCars', 'GarageArea', 'WoodDeckSF', 'OpenPorchSF', 'EnclosedPorch', '3SsnPorch', 'ScreenPorch', 'PoolArea', 'MiscVal', 'MoSold', 'YrSold']</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="8409ac94-ef05-4c64-94ec-bdc12c02a60e" type="checkbox" ><label class="sk-toggleable__label" for="8409ac94-ef05-4c64-94ec-bdc12c02a60e">SimpleImputer</label><div class="sk-toggleable__content"><pre>SimpleImputer(strategy='constant')</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="c21ae064-839b-4b16-98af-3893cdaaf4a6" type="checkbox" ><label class="sk-toggleable__label" for="c21ae064-839b-4b16-98af-3893cdaaf4a6">StandardScaler</label><div class="sk-toggleable__content"><pre>StandardScaler()</pre></div></div></div></div></div></div></div></div><div class="sk-parallel-item"><div class="sk-item"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="a236afdf-86ee-41f6-ae11-29321832dffc" type="checkbox" ><label class="sk-toggleable__label" for="a236afdf-86ee-41f6-ae11-29321832dffc">cat_cols</label><div class="sk-toggleable__content"><pre>['MSZoning', 'Street', 'Alley', 'LotShape', 'LandContour', 'Utilities', 'LotConfig', 'LandSlope', 'Condition1', 'Condition2', 'BldgType', 'HouseStyle', 'RoofStyle', 'RoofMatl', 'MasVnrType', 'ExterQual', 'ExterCond', 'Foundation', 'BsmtQual', 'BsmtCond', 'BsmtExposure', 'BsmtFinType1', 'BsmtFinType2', 'Heating', 'HeatingQC', 'CentralAir', 'Electrical', 'KitchenQual', 'Functional', 'FireplaceQu', 'GarageType', 'GarageFinish', 'GarageQual', 'GarageCond', 'PavedDrive', 'PoolQC', 'Fence', 'MiscFeature', 'SaleType', 'SaleCondition', 'Neighborhood', 'Exterior1st', 'Exterior2nd']</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="9678d35d-28f7-4def-95f0-de5c8886f164" type="checkbox" ><label class="sk-toggleable__label" for="9678d35d-28f7-4def-95f0-de5c8886f164">SimpleImputer</label><div class="sk-toggleable__content"><pre>SimpleImputer(strategy='constant')</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="035300a4-0bb7-4c10-bfe8-83b6437dbafd" type="checkbox" ><label class="sk-toggleable__label" for="035300a4-0bb7-4c10-bfe8-83b6437dbafd">OneHotEncoder</label><div class="sk-toggleable__content"><pre>OneHotEncoder(handle_unknown='ignore')</pre></div></div></div></div></div></div></div></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="a16d44b4-053d-4443-aea3-d6869cec4d52" type="checkbox" ><label class="sk-toggleable__label" for="a16d44b4-053d-4443-aea3-d6869cec4d52">XGBRegressor</label><div class="sk-toggleable__content"><pre>XGBRegressor(base_score=None, booster=None, colsample_bylevel=None,
colsample_bynode=None, colsample_bytree=None,
enable_categorical=False, eval_metric='mae', gamma=None,
gpu_id=None, importance_type=None, interaction_constraints=None,
learning_rate=None, max_delta_step=None, max_depth=None,
min_child_weight=None, missing=nan, monotone_constraints=None,
n_estimators=100, n_jobs=11, num_parallel_tree=None,
predictor=None, random_state=42, reg_alpha=None, reg_lambda=None,
scale_pos_weight=None, subsample=None, tree_method=None,
use_label_encoder=False, validate_parameters=None, verbosity=None)</pre></div></div></div></div></div></div></div>
# create train, valid split to evaulate model on valid dataset
tmlt.dfl.create_train_valid(valid_size=0.2)
start = time.time()
# Now fit
tmlt.spl.fit(tmlt.dfl.X_train, tmlt.dfl.y_train)
end = time.time()
print("Fit Time:", end - start)
#predict
preds = tmlt.spl.predict(tmlt.dfl.X_valid)
print('X_valid MAE:', mean_absolute_error(tmlt.dfl.y_valid, preds))
Fit Time: 0.232680082321167
X_valid MAE: 16565.101415346748
In background prepare_data_for_training method loads your input data into Pandas DataFrame, seprates X(features) and y(target).
The prepare_data_for_training methods prepare X and y DataFrames, preprocess all numerical and categorical type data found in these DataFrames using scikit-learn pipelines. Then it bundle preprocessed data with your given model and return an MLPipeline object, this class instance has dataframeloader, preprocessor and scikit-lean pipeline instances.
The create_train_valid method use valid_size to split X(features) into X_train, y_train, X_valid and y_valid DataFrames, so you can call fit methods on X_train and y_train and predict methods on X_valid or X_test.
Please check detail documentation and source code for more details.
NOTE: If you want to customize data and preprocessing steps you can do so by using DataFrameLoader and PreProessor classes. Check detail documentations for these classes for more options.
To see more clear picture of model performance, Let's do a quick Cross Validation on our Pipeline
start = time.time()
# Now do cross_validation
scores = tmlt.do_cross_validation(cv=5, scoring='neg_mean_absolute_error')
end = time.time()
print("Cross Validation Time:", end - start)
print("scores:", scores)
print("Average MAE score:", scores.mean())
Cross Validation Time: 1.1889641284942627
scores: [15865.5140732 18431.08924176 18670.3333155 15439.67369435
16835.6969847 ]
Average MAE score: 17048.461461900682
MAE did came out slightly bad with cross validation
Let's see if we can improve our cross validation score with hyperparams tunning
# lets choose our choice of metrics
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score, mean_squared_log_error
we are using optuna based hyperparameter search here, make sure to supply a new directory path so search is saved
study = tmlt.do_xgb_optuna_optimization(preds_metrics=[mean_absolute_error,
mean_squared_error,
r2_score],
output_dir_path=OUTPUT_PATH)
print(study.best_trial)
2021-11-22 16:58:20,148 INFO Optimization Direction is: minimize
[32m[I 2021-11-22 16:58:20,175][0m Using an existing study with name 'tmlt_autoxgb' instead of creating a new one.[0m
2021-11-22 16:58:20,296 INFO Training Started!
[16:58:20] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "colsample_bytree", "max_depth", "subsample", "tree_method" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 16:58:26,066 INFO Training Ended!
2021-11-22 16:58:26,116 INFO mean_absolute_error: 18430.128959760274
2021-11-22 16:58:26,116 INFO mean_squared_error: 1003740151.2011865
2021-11-22 16:58:26,117 INFO r2_score: 0.8691398352444675
[32m[I 2021-11-22 16:58:26,148][0m Trial 5 finished with value: 1003740151.2011865 and parameters: {'learning_rate': 0.011287646791421295, 'reg_lambda': 0.06938986879711422, 'reg_alpha': 8.176124256580969e-05, 'subsample': 0.3654130556583419, 'colsample_bytree': 0.3647997317624503, 'max_depth': 1, 'early_stopping_rounds': 240, 'n_estimators': 20000, 'tree_method': 'approx', 'booster': 'gblinear'}. Best is trial 2 with value: 859935950.534274.[0m
2021-11-22 16:58:26,247 INFO Training Started!
[16:58:26] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "colsample_bytree", "max_depth", "subsample", "tree_method" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 16:58:28,378 INFO Training Ended!
2021-11-22 16:58:28,424 INFO mean_absolute_error: 31222.505752354453
2021-11-22 16:58:28,424 INFO mean_squared_error: 2771208996.3631716
2021-11-22 16:58:28,425 INFO r2_score: 0.638710411850993
[32m[I 2021-11-22 16:58:28,449][0m Trial 6 finished with value: 2771208996.3631716 and parameters: {'learning_rate': 0.03334511464699647, 'reg_lambda': 5.25800680500751, 'reg_alpha': 0.045533012584301466, 'subsample': 0.5010774952608195, 'colsample_bytree': 0.5190423499031988, 'max_depth': 5, 'early_stopping_rounds': 122, 'n_estimators': 7000, 'tree_method': 'hist', 'booster': 'gblinear'}. Best is trial 2 with value: 859935950.534274.[0m
2021-11-22 16:58:28,542 INFO Training Started!
[16:58:28] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "colsample_bytree", "max_depth", "subsample", "tree_method" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 16:58:34,604 INFO Training Ended!
2021-11-22 16:58:34,657 INFO mean_absolute_error: 18308.20339255137
2021-11-22 16:58:34,657 INFO mean_squared_error: 819804093.191586
2021-11-22 16:58:34,658 INFO r2_score: 0.8931200484767615
[32m[I 2021-11-22 16:58:34,683][0m Trial 7 finished with value: 819804093.191586 and parameters: {'learning_rate': 0.04089996575161834, 'reg_lambda': 8.596335607070313e-05, 'reg_alpha': 0.02043920450683123, 'subsample': 0.9184407885806762, 'colsample_bytree': 0.20385362758786968, 'max_depth': 9, 'early_stopping_rounds': 485, 'n_estimators': 20000, 'tree_method': 'exact', 'booster': 'gblinear'}. Best is trial 7 with value: 819804093.191586.[0m
2021-11-22 16:58:34,804 INFO Training Started!
2021-11-22 16:59:17,739 INFO Training Ended!
2021-11-22 16:59:18,499 INFO mean_absolute_error: 16149.496602097603
2021-11-22 16:59:18,500 INFO mean_squared_error: 842794160.6512825
2021-11-22 16:59:18,500 INFO r2_score: 0.8901227747182928
[32m[I 2021-11-22 16:59:18,532][0m Trial 8 finished with value: 842794160.6512825 and parameters: {'learning_rate': 0.030222339821538807, 'reg_lambda': 0.09895705776538985, 'reg_alpha': 2.2504091517410305e-06, 'subsample': 0.5584749705539753, 'colsample_bytree': 0.15912009415082107, 'max_depth': 9, 'early_stopping_rounds': 500, 'n_estimators': 20000, 'tree_method': 'exact', 'booster': 'gbtree', 'gamma': 9.917098787836175e-07, 'grow_policy': 'lossguide'}. Best is trial 7 with value: 819804093.191586.[0m
2021-11-22 16:59:18,717 INFO Training Started!
[16:59:18] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "colsample_bytree", "max_depth", "subsample", "tree_method" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 16:59:23,129 INFO Training Ended!
2021-11-22 16:59:23,177 INFO mean_absolute_error: 18419.340887200342
2021-11-22 16:59:23,178 INFO mean_squared_error: 817311608.0615362
2021-11-22 16:59:23,178 INFO r2_score: 0.8934450001232396
[32m[I 2021-11-22 16:59:23,208][0m Trial 9 finished with value: 817311608.0615362 and parameters: {'learning_rate': 0.22322780828302763, 'reg_lambda': 1.6754489349191887e-05, 'reg_alpha': 2.1992031654524757e-06, 'subsample': 0.5257239751043923, 'colsample_bytree': 0.1836836077039992, 'max_depth': 6, 'early_stopping_rounds': 442, 'n_estimators': 15000, 'tree_method': 'approx', 'booster': 'gblinear'}. Best is trial 9 with value: 817311608.0615362.[0m
2021-11-22 16:59:23,321 INFO Training Started!
[16:59:23] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "colsample_bytree", "max_depth", "subsample", "tree_method" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 16:59:28,052 INFO Training Ended!
2021-11-22 16:59:28,104 INFO mean_absolute_error: 56409.48624785959
2021-11-22 16:59:28,105 INFO mean_squared_error: 6504554246.374722
2021-11-22 16:59:28,105 INFO r2_score: 0.15198466523106557
[32m[I 2021-11-22 16:59:28,137][0m Trial 10 finished with value: 6504554246.374722 and parameters: {'learning_rate': 0.1340147043391367, 'reg_lambda': 63.39872490630416, 'reg_alpha': 6.373289028896017e-05, 'subsample': 0.4471511816218636, 'colsample_bytree': 0.20334039219317368, 'max_depth': 4, 'early_stopping_rounds': 101, 'n_estimators': 15000, 'tree_method': 'hist', 'booster': 'gblinear'}. Best is trial 9 with value: 817311608.0615362.[0m
2021-11-22 16:59:28,237 INFO Training Started!
[16:59:28] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "colsample_bytree", "max_depth", "subsample", "tree_method" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 16:59:35,476 INFO Training Ended!
2021-11-22 16:59:35,527 INFO mean_absolute_error: 18619.747498394692
2021-11-22 16:59:35,528 INFO mean_squared_error: 1060765653.880221
2021-11-22 16:59:35,528 INFO r2_score: 0.8617052749482441
[32m[I 2021-11-22 16:59:35,552][0m Trial 11 finished with value: 1060765653.880221 and parameters: {'learning_rate': 0.018948988994948266, 'reg_lambda': 0.11940360049263372, 'reg_alpha': 0.3478769379762252, 'subsample': 0.19267541567301605, 'colsample_bytree': 0.46957499489616195, 'max_depth': 3, 'early_stopping_rounds': 390, 'n_estimators': 20000, 'tree_method': 'exact', 'booster': 'gblinear'}. Best is trial 9 with value: 817311608.0615362.[0m
2021-11-22 16:59:35,686 INFO Training Started!
2021-11-22 17:00:28,688 INFO Training Ended!
2021-11-22 17:00:28,888 INFO mean_absolute_error: 16017.475412029109
2021-11-22 17:00:28,888 INFO mean_squared_error: 731863525.6978335
2021-11-22 17:00:28,889 INFO r2_score: 0.9045850846588409
[32m[I 2021-11-22 17:00:28,915][0m Trial 12 finished with value: 731863525.6978335 and parameters: {'learning_rate': 0.07808709699682324, 'reg_lambda': 9.871089998565025e-08, 'reg_alpha': 1.0673222786209778e-08, 'subsample': 0.7165371749773521, 'colsample_bytree': 0.7641468442580719, 'max_depth': 7, 'early_stopping_rounds': 376, 'n_estimators': 15000, 'tree_method': 'approx', 'booster': 'gbtree', 'gamma': 0.8079088948323129, 'grow_policy': 'lossguide'}. Best is trial 12 with value: 731863525.6978335.[0m
2021-11-22 17:00:29,106 INFO Training Started!
2021-11-22 17:01:16,207 INFO Training Ended!
2021-11-22 17:01:16,377 INFO mean_absolute_error: 16352.778454088186
2021-11-22 17:01:16,378 INFO mean_squared_error: 744122288.9684557
2021-11-22 17:01:16,379 INFO r2_score: 0.9029868784105675
[32m[I 2021-11-22 17:01:16,409][0m Trial 13 finished with value: 744122288.9684557 and parameters: {'learning_rate': 0.08318544049038572, 'reg_lambda': 4.654274233615053e-08, 'reg_alpha': 2.184767986189026e-08, 'subsample': 0.6746291410083555, 'colsample_bytree': 0.7990169757948151, 'max_depth': 7, 'early_stopping_rounds': 381, 'n_estimators': 15000, 'tree_method': 'approx', 'booster': 'gbtree', 'gamma': 0.4585867037796719, 'grow_policy': 'lossguide'}. Best is trial 12 with value: 731863525.6978335.[0m
2021-11-22 17:01:16,594 INFO Training Started!
2021-11-22 17:02:07,158 INFO Training Ended!
2021-11-22 17:02:07,335 INFO mean_absolute_error: 16126.497953232021
2021-11-22 17:02:07,336 INFO mean_squared_error: 780217945.1506925
2021-11-22 17:02:07,337 INFO r2_score: 0.8982809956088145
[32m[I 2021-11-22 17:02:07,365][0m Trial 14 finished with value: 780217945.1506925 and parameters: {'learning_rate': 0.07549860943857781, 'reg_lambda': 1.7749082269700017e-08, 'reg_alpha': 1.941329606118197e-08, 'subsample': 0.736007047743309, 'colsample_bytree': 0.8737589345624964, 'max_depth': 7, 'early_stopping_rounds': 357, 'n_estimators': 15000, 'tree_method': 'approx', 'booster': 'gbtree', 'gamma': 0.8158952450888562, 'grow_policy': 'lossguide'}. Best is trial 12 with value: 731863525.6978335.[0m
2021-11-22 17:02:07,553 INFO Training Started!
2021-11-22 17:02:47,669 INFO Training Ended!
2021-11-22 17:02:48,209 INFO mean_absolute_error: 15801.584706763699
2021-11-22 17:02:48,210 INFO mean_squared_error: 766536423.2441648
2021-11-22 17:02:48,211 INFO r2_score: 0.9000646905309038
[32m[I 2021-11-22 17:02:48,247][0m Trial 15 finished with value: 766536423.2441648 and parameters: {'learning_rate': 0.07398928435966287, 'reg_lambda': 1.1940556339241501e-08, 'reg_alpha': 1.760152827513985e-08, 'subsample': 0.7195078189746692, 'colsample_bytree': 0.6899120607572461, 'max_depth': 7, 'early_stopping_rounds': 335, 'n_estimators': 15000, 'tree_method': 'approx', 'booster': 'gbtree', 'gamma': 0.008291443066176078, 'grow_policy': 'lossguide'}. Best is trial 12 with value: 731863525.6978335.[0m
2021-11-22 17:02:48,427 INFO Training Started!
2021-11-22 17:03:35,781 INFO Training Ended!
2021-11-22 17:03:36,139 INFO mean_absolute_error: 15482.647688356165
2021-11-22 17:03:36,140 INFO mean_squared_error: 686933953.1421537
2021-11-22 17:03:36,140 INFO r2_score: 0.9104426676797013
[32m[I 2021-11-22 17:03:36,166][0m Trial 16 finished with value: 686933953.1421537 and parameters: {'learning_rate': 0.08235476908327058, 'reg_lambda': 6.647799361804065e-07, 'reg_alpha': 1.0067601266720434e-08, 'subsample': 0.7228884546525054, 'colsample_bytree': 0.9947016819536298, 'max_depth': 7, 'early_stopping_rounds': 409, 'n_estimators': 15000, 'tree_method': 'approx', 'booster': 'gbtree', 'gamma': 0.0038226799902060316, 'grow_policy': 'lossguide'}. Best is trial 16 with value: 686933953.1421537.[0m
2021-11-22 17:03:36,332 INFO Training Started!
2021-11-22 17:03:55,399 INFO Training Ended!
2021-11-22 17:03:55,565 INFO mean_absolute_error: 16349.99369916524
2021-11-22 17:03:55,566 INFO mean_squared_error: 807699951.7007654
2021-11-22 17:03:55,567 INFO r2_score: 0.8946980962890538
[32m[I 2021-11-22 17:03:55,600][0m Trial 17 finished with value: 807699951.7007654 and parameters: {'learning_rate': 0.055596616114553216, 'reg_lambda': 1.3893364258209366e-06, 'reg_alpha': 67.41099834562895, 'subsample': 0.8192815751548503, 'colsample_bytree': 0.9860680008198711, 'max_depth': 8, 'early_stopping_rounds': 427, 'n_estimators': 15000, 'tree_method': 'approx', 'booster': 'gbtree', 'gamma': 0.001304182841715915, 'grow_policy': 'lossguide'}. Best is trial 16 with value: 686933953.1421537.[0m
2021-11-22 17:03:55,770 INFO Training Started!
2021-11-22 17:04:11,003 INFO Training Ended!
2021-11-22 17:04:11,158 INFO mean_absolute_error: 15908.28640036387
2021-11-22 17:04:11,159 INFO mean_squared_error: 778115201.6020439
2021-11-22 17:04:11,160 INFO r2_score: 0.8985551356508219
[32m[I 2021-11-22 17:04:11,188][0m Trial 18 finished with value: 778115201.6020439 and parameters: {'learning_rate': 0.09711661764783566, 'reg_lambda': 7.823454743474081e-07, 'reg_alpha': 4.5072692770014974e-07, 'subsample': 0.6263107892204567, 'colsample_bytree': 0.9527217430143557, 'max_depth': 5, 'early_stopping_rounds': 318, 'n_estimators': 7000, 'tree_method': 'approx', 'booster': 'gbtree', 'gamma': 1.117709416173302e-08, 'grow_policy': 'lossguide'}. Best is trial 16 with value: 686933953.1421537.[0m
2021-11-22 17:04:11,326 INFO Training Started!
2021-11-22 17:04:40,256 INFO Training Ended!
2021-11-22 17:04:40,573 INFO mean_absolute_error: 15689.446797410103
2021-11-22 17:04:40,574 INFO mean_squared_error: 716682978.005267
2021-11-22 17:04:40,574 INFO r2_score: 0.9065642113977197
[32m[I 2021-11-22 17:04:40,608][0m Trial 19 finished with value: 716682978.005267 and parameters: {'learning_rate': 0.0578712823399666, 'reg_lambda': 1.1030332244677372e-06, 'reg_alpha': 4.487986572959091e-06, 'subsample': 0.8699561697539606, 'colsample_bytree': 0.6341900847864899, 'max_depth': 6, 'early_stopping_rounds': 419, 'n_estimators': 15000, 'tree_method': 'approx', 'booster': 'gbtree', 'gamma': 0.005851046483654154, 'grow_policy': 'lossguide'}. Best is trial 16 with value: 686933953.1421537.[0m
FrozenTrial(number=16, values=[686933953.1421537], datetime_start=datetime.datetime(2021, 11, 22, 17, 2, 48, 252811), datetime_complete=datetime.datetime(2021, 11, 22, 17, 3, 36, 141725), params={'booster': 'gbtree', 'colsample_bytree': 0.9947016819536298, 'early_stopping_rounds': 409, 'gamma': 0.0038226799902060316, 'grow_policy': 'lossguide', 'learning_rate': 0.08235476908327058, 'max_depth': 7, 'n_estimators': 15000, 'reg_alpha': 1.0067601266720434e-08, 'reg_lambda': 6.647799361804065e-07, 'subsample': 0.7228884546525054, 'tree_method': 'approx'}, distributions={'booster': CategoricalDistribution(choices=('gbtree', 'gblinear')), 'colsample_bytree': UniformDistribution(high=1.0, low=0.1), 'early_stopping_rounds': IntUniformDistribution(high=500, low=100, step=1), 'gamma': LogUniformDistribution(high=1.0, low=1e-08), 'grow_policy': CategoricalDistribution(choices=('depthwise', 'lossguide')), 'learning_rate': LogUniformDistribution(high=0.25, low=0.01), 'max_depth': IntUniformDistribution(high=9, low=1, step=1), 'n_estimators': CategoricalDistribution(choices=(7000, 15000, 20000)), 'reg_alpha': LogUniformDistribution(high=100.0, low=1e-08), 'reg_lambda': LogUniformDistribution(high=100.0, low=1e-08), 'subsample': UniformDistribution(high=1.0, low=0.1), 'tree_method': CategoricalDistribution(choices=('exact', 'approx', 'hist'))}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE, value=None)
Let's use our newly found best params to update the model on sklearn pipeline
xgb_params.update(study.best_trial.params)
print("xgb_params", xgb_params)
xgb_model = XGBRegressor(**xgb_params)
tmlt.update_model(xgb_model)
tmlt.spl
xgb_params {'use_label_encoder': False, 'eval_metric': 'mae', 'random_state': 42, 'booster': 'gbtree', 'colsample_bytree': 0.9947016819536298, 'early_stopping_rounds': 409, 'gamma': 0.0038226799902060316, 'grow_policy': 'lossguide', 'learning_rate': 0.08235476908327058, 'max_depth': 7, 'n_estimators': 15000, 'reg_alpha': 1.0067601266720434e-08, 'reg_lambda': 6.647799361804065e-07, 'subsample': 0.7228884546525054, 'tree_method': 'approx'}
<style>#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a {color: black;background-color: white;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a pre{padding: 0;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-toggleable {background-color: white;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-estimator:hover {background-color: #d4ebff;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-parallel-item::after {content: "";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-serial::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 2em;bottom: 0;left: 50%;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-item {z-index: 1;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-parallel::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 2em;bottom: 0;left: 50%;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-parallel-item {display: flex;flex-direction: column;position: relative;background-color: white;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-parallel-item:only-child::after {width: 0;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;position: relative;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-label label {font-family: monospace;font-weight: bold;background-color: white;display: inline-block;line-height: 1.2em;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-label-container {position: relative;z-index: 2;text-align: center;}#sk-2e31a1db-0287-4ccc-86d3-47f28ee2ee9a div.sk-container {display: inline-block;position: relative;}</style>PipelinePipeline(steps=[('preprocessor',
ColumnTransformer(transformers=[('num_cols',
Pipeline(steps=[('imputer',
SimpleImputer(strategy='constant')),
('scaler',
StandardScaler())]),
['MSSubClass', 'LotFrontage',
'LotArea', 'OverallQual',
'OverallCond', 'YearBuilt',
'YearRemodAdd', 'MasVnrArea',
'BsmtFinSF1', 'BsmtFinSF2',
'BsmtUnfSF', 'TotalBsmtSF',
'1stFlrSF', '2ndFlrSF',
'...
learning_rate=0.08235476908327058,
max_delta_step=None, max_depth=7,
min_child_weight=None, missing=nan,
monotone_constraints=None, n_estimators=15000,
n_jobs=None, num_parallel_tree=None,
predictor=None, random_state=42,
reg_alpha=1.0067601266720434e-08,
reg_lambda=6.647799361804065e-07,
scale_pos_weight=None,
subsample=0.7228884546525054,
tree_method='approx', use_label_encoder=False, ...))])</pre></div></div></div><div class="sk-serial"><div class="sk-item sk-dashed-wrapped"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="0c96efa7-77dd-4ed9-b87f-c46dc5ec64ee" type="checkbox" ><label class="sk-toggleable__label" for="0c96efa7-77dd-4ed9-b87f-c46dc5ec64ee">preprocessor: ColumnTransformer</label><div class="sk-toggleable__content"><pre>ColumnTransformer(transformers=[('num_cols',
Pipeline(steps=[('imputer',
SimpleImputer(strategy='constant')),
('scaler', StandardScaler())]),
['MSSubClass', 'LotFrontage', 'LotArea',
'OverallQual', 'OverallCond', 'YearBuilt',
'YearRemodAdd', 'MasVnrArea', 'BsmtFinSF1',
'BsmtFinSF2', 'BsmtUnfSF', 'TotalBsmtSF',
'1stFlrSF', '2ndFlrSF', 'LowQualFinSF',
'GrLivArea', 'BsmtF...
['MSZoning', 'Street', 'Alley', 'LotShape',
'LandContour', 'Utilities', 'LotConfig',
'LandSlope', 'Condition1', 'Condition2',
'BldgType', 'HouseStyle', 'RoofStyle',
'RoofMatl', 'MasVnrType', 'ExterQual',
'ExterCond', 'Foundation', 'BsmtQual',
'BsmtCond', 'BsmtExposure', 'BsmtFinType1',
'BsmtFinType2', 'Heating', 'HeatingQC',
'CentralAir', 'Electrical', 'KitchenQual',
'Functional', 'FireplaceQu', ...])])</pre></div></div></div><div class="sk-parallel"><div class="sk-parallel-item"><div class="sk-item"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="a236f6ea-f75b-4ac9-8a89-322e2e8089fa" type="checkbox" ><label class="sk-toggleable__label" for="a236f6ea-f75b-4ac9-8a89-322e2e8089fa">num_cols</label><div class="sk-toggleable__content"><pre>['MSSubClass', 'LotFrontage', 'LotArea', 'OverallQual', 'OverallCond', 'YearBuilt', 'YearRemodAdd', 'MasVnrArea', 'BsmtFinSF1', 'BsmtFinSF2', 'BsmtUnfSF', 'TotalBsmtSF', '1stFlrSF', '2ndFlrSF', 'LowQualFinSF', 'GrLivArea', 'BsmtFullBath', 'BsmtHalfBath', 'FullBath', 'HalfBath', 'BedroomAbvGr', 'KitchenAbvGr', 'TotRmsAbvGrd', 'Fireplaces', 'GarageYrBlt', 'GarageCars', 'GarageArea', 'WoodDeckSF', 'OpenPorchSF', 'EnclosedPorch', '3SsnPorch', 'ScreenPorch', 'PoolArea', 'MiscVal', 'MoSold', 'YrSold']</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="02562a0c-2fbf-46a9-a451-c0b32d0dadea" type="checkbox" ><label class="sk-toggleable__label" for="02562a0c-2fbf-46a9-a451-c0b32d0dadea">SimpleImputer</label><div class="sk-toggleable__content"><pre>SimpleImputer(strategy='constant')</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="80b365be-3573-4be7-9375-42eaab4e66ab" type="checkbox" ><label class="sk-toggleable__label" for="80b365be-3573-4be7-9375-42eaab4e66ab">StandardScaler</label><div class="sk-toggleable__content"><pre>StandardScaler()</pre></div></div></div></div></div></div></div></div><div class="sk-parallel-item"><div class="sk-item"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="caea1be7-b864-4a91-b796-607b663c9e49" type="checkbox" ><label class="sk-toggleable__label" for="caea1be7-b864-4a91-b796-607b663c9e49">cat_cols</label><div class="sk-toggleable__content"><pre>['MSZoning', 'Street', 'Alley', 'LotShape', 'LandContour', 'Utilities', 'LotConfig', 'LandSlope', 'Condition1', 'Condition2', 'BldgType', 'HouseStyle', 'RoofStyle', 'RoofMatl', 'MasVnrType', 'ExterQual', 'ExterCond', 'Foundation', 'BsmtQual', 'BsmtCond', 'BsmtExposure', 'BsmtFinType1', 'BsmtFinType2', 'Heating', 'HeatingQC', 'CentralAir', 'Electrical', 'KitchenQual', 'Functional', 'FireplaceQu', 'GarageType', 'GarageFinish', 'GarageQual', 'GarageCond', 'PavedDrive', 'PoolQC', 'Fence', 'MiscFeature', 'SaleType', 'SaleCondition', 'Neighborhood', 'Exterior1st', 'Exterior2nd']</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="a08af16a-85b9-4e86-89d7-8b5c4c96d948" type="checkbox" ><label class="sk-toggleable__label" for="a08af16a-85b9-4e86-89d7-8b5c4c96d948">SimpleImputer</label><div class="sk-toggleable__content"><pre>SimpleImputer(strategy='constant')</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="6c519629-2d6c-4b15-b612-5e18d611fbf6" type="checkbox" ><label class="sk-toggleable__label" for="6c519629-2d6c-4b15-b612-5e18d611fbf6">OneHotEncoder</label><div class="sk-toggleable__content"><pre>OneHotEncoder(handle_unknown='ignore')</pre></div></div></div></div></div></div></div></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="098bbbcd-22b3-409d-940e-b0e16376c24c" type="checkbox" ><label class="sk-toggleable__label" for="098bbbcd-22b3-409d-940e-b0e16376c24c">XGBRegressor</label><div class="sk-toggleable__content"><pre>XGBRegressor(base_score=None, booster='gbtree', colsample_bylevel=None,
colsample_bynode=None, colsample_bytree=0.9947016819536298,
early_stopping_rounds=409, enable_categorical=False,
eval_metric='mae', gamma=0.0038226799902060316, gpu_id=None,
grow_policy='lossguide', importance_type=None,
interaction_constraints=None, learning_rate=0.08235476908327058,
max_delta_step=None, max_depth=7, min_child_weight=None,
missing=nan, monotone_constraints=None, n_estimators=15000,
n_jobs=None, num_parallel_tree=None, predictor=None,
random_state=42, reg_alpha=1.0067601266720434e-08,
reg_lambda=6.647799361804065e-07, scale_pos_weight=None,
subsample=0.7228884546525054, tree_method='approx',
use_label_encoder=False, ...)</pre></div></div></div></div></div></div></div>
Now, Let's use 5 K-Fold Training on this Updated XGB model with best params found from Optuna search
# k-fold training
xgb_model_metrics_score, xgb_model_test_preds = tmlt.do_kfold_training(n_splits=5, metrics=mean_absolute_error)
/Users/pamathur/miniconda3/envs/nbdev_env/lib/python3.9/site-packages/sklearn/model_selection/_split.py:676: UserWarning: The least populated class in y has only 1 members, which is less than n_splits=10.
warnings.warn(
[17:05:00] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "early_stopping_rounds" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 17:05:47,618 INFO fold: 1 , mean_absolute_error: 15661.41831656678
[17:05:47] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "early_stopping_rounds" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 17:06:59,831 INFO fold: 2 , mean_absolute_error: 16383.731324914384
[17:06:59] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "early_stopping_rounds" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 17:07:50,938 INFO fold: 3 , mean_absolute_error: 15294.39223030822
[17:07:50] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "early_stopping_rounds" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 17:08:47,789 INFO fold: 4 , mean_absolute_error: 14800.260916095891
[17:08:47] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "early_stopping_rounds" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 17:09:37,629 INFO fold: 5 , mean_absolute_error: 14129.463800299658
[17:09:37] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "early_stopping_rounds" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 17:10:28,624 INFO fold: 6 , mean_absolute_error: 19015.87021083048
[17:10:28] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "early_stopping_rounds" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 17:11:37,261 INFO fold: 7 , mean_absolute_error: 14327.51765839041
[17:11:37] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "early_stopping_rounds" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 17:12:48,462 INFO fold: 8 , mean_absolute_error: 17722.195740582192
[17:12:48] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "early_stopping_rounds" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 17:13:39,624 INFO fold: 9 , mean_absolute_error: 15262.005136986301
[17:13:39] WARNING: /Users/runner/miniforge3/conda-bld/xgboost-split_1634712680264/work/src/learner.cc:576:
Parameters: { "early_stopping_rounds" } might not be used.
This could be a false alarm, with some parameters getting used by language bindings but
then being mistakenly passed down to XGBoost core, or some parameter actually being used
but getting flagged wrongly here. Please open an issue if you find any such cases.
2021-11-22 17:14:30,893 INFO fold: 10 , mean_absolute_error: 15830.971532534246
2021-11-22 17:14:30,894 INFO mean metrics score: 15842.782686750856
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/var/folders/p3/zmg8jfwx0hb9gwzs0w69d7f0rgyjx2/T/ipykernel_70698/3966110186.py in <module>
2 xgb_model_metrics_score, xgb_model_test_preds = tmlt.do_kfold_training(n_splits=10, metrics=mean_absolute_error)
3 # predict on test dataset
----> 4 if xgb_model_test_preds:
5 print(xgb_model_preds.shape)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
# predict on test dataset
if xgb_model_test_preds is not None:
print(xgb_model_test_preds.shape)
(1459,)
You can even improve metrics score further by running Optuna search for longer time or rerunning the study, check documentation for more details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tabular_ml_toolkit-0.0.18.tar.gz
(50.5 kB
view hashes)
Built Distribution
Close
Hashes for tabular_ml_toolkit-0.0.18.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | d1e3b4918f6ee6414e6ca5d184214c6f875dc461402ff398eee769d9569cb49a |
|
| MD5 | c21f3255d32b0b95b9057bbaef23bde5 |
|
| BLAKE2b-256 | 58f15e5dd662caa6a87676b3ede95223741bafa9995a70104d5add224361713b |
Close
Hashes for tabular_ml_toolkit-0.0.18-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 157fa358f2c211e3fb91327b0d90ab625c31c9e1eb882bd44dd24b224a1ac51a |
|
| MD5 | 6cc2f57940d635abcfc53e878e1385e8 |
|
| BLAKE2b-256 | a2f23f829f7a5563734ffe8094408345a0fc73c0a8471645081546a760495a03 |
