GBART

Introduction

GBART is a pure python package to implement our proposed algorithm GBART in our ICASSP2020 submitted paper:

Variable Grouping based Bayesian Additive Regression Tree

Through GBART, We will seek for potential grouping of variables in such way that there is no nonlinear interaction term between variables of different groups, then we build the final model that can take advantage of such concealed information.

GBART consists of two stages. The first stage is to search for potential interactions and an appropriate grouping of variables. The second stage is to build a final model based on the discovered groups.

This python package is built based on BartPy package, the pure python version of BART proposed by Chipman et. al [1]. For details please refer to the offical website BartPy.

Installation

Using pip

pip install gbart
Note: a typo is fixing, somehow using pip will result in a missed subfolder(modified_bart), we recommend install it manually.

Mannually installation

1. Visit our website https://github.com/augusHsu/gbart

2. Directly download folder "gbart"

3. put folders into place where you usually import packages, generally, it should be in 
 ~/bin/python3.7/site-packages/ 
 Or put it with your script in same folder, then it will be fine.

Usage and examples

The easiest way to run and obtain result (generated data)

• Preparation

import copy
import numpy as np
import gbart.utilities as ut  # provide helper functions
import gbart.create_dataset as cd  # data generator
from gbart.groupbart import * 

• generate dataset

dataset = cd.create_friedman()
# the last column of dataset is dependent variable, the output Y.

• Build model and get accuracy by using BART.

acc_o = build_original_model(dataset)
# This function splits the whole dataset into training and testing (80% for training)
# This function returns accuracy in testing data.

• Get the grouping information

output_pair = get_pair(dataset, theta=1)
# return the grouping information. The first stage of GBART.
# theta is a parameter that controls variable selection. Suggested range is [0,1]
# To turn off the variable selection, set theta=0
# More variables are likely to be removed when theta is higher.

• Build the gbart model

acc_g = build_group_wise_model(dataset, output_pair)
# take "output_pair" as pair_list 
# This function returns accuracy in testing data.

To design a customization version and/or tune model parameters. Please consider the following

• Write your own helper function instead of calling functions in groupbart.py, the only thing you may need in groupbart.py is get_pair(dataset), which will help you find the proper grouping information.

• Tune the parameters of both BART and GBART model. Here is an easy example.

import numpy as np
import gbart.utilities as ut
from gbart.modified_bartpy.sklearnmodel import SklearnModel

# Data preparation 
b = int(0.8 * np.shape(dataset)[0])  
Data_train = dataset[:b,:]
Data_predict = dataset[b:,:]
x_data = Data_train[:,:-1]
y_data = Data_train[:,-1]


# Building the model
model = SklearnModel(sublist=None,
                     n_trees=50,
                     n_chains=4,
                     n_samples=50,
                     n_burn=200,
                     thin=0.1,
                     n_jobs=1)
# This GBART model is built based on BART model, a new feature named sublist is added. 
# sublist can either take "None" or list of groups of variables as input.
# when sublist is None, it will be the exact BART model.
# when sublist is a list, it will build GBART model.
# To tune other parameters, please refer to BartPy usage guide.

# fit and prediction 
model.fit(x_data, y_data)
y_pred = model.predict(Data_predict[:,:-1])
y_true = Data_predict[:,-1]
acc = ut.get_error_reg(y_pred, y_true)

Real data example

• Preparation

import numpy as np
import pandas as pd
from gbart.groupbart import * 

• load and pre-processing data

url = "~/Admission_Predict_Ver1.1.csv"
data = pd.read_csv(url)
data = data.drop(columns="Serial No.")
dataset = data.iloc[:, :].values
# Note our package can match with Sklearn, we use the same dataset type. 

• Build model and get accuracy by using BART.

acc_o = build_original_model(dataset)
# This function splits the whole dataset into training and testing (80% for training)
# This function returns accuracy in testing data.

• Get the grouping information

output_pair = get_pair(dataset)
# return the the grouping information. The first phase of Gbart algorithm.

• Build the gbart model

acc_g = build_group_wise_model(dataset, output_pair)
# take "output_pair" as pair_list 
# This function returns accuracy in testing data.

• Tune the parameters of both BART and GBART model. Please refer to the generated part.

• One more advcanced thing that worth to explore: During the group searching process, we may need to set some threshold to control the variable selection. I fix it to be 85 percent of the best, you can tune it by change that constant to be an attribute in get_pair(dataset)

Acknowledge

We truly thank Mr. Jake Coltman for his contribution to BartPy package.

Reference

[1] Chipman, Hugh A., Edward I. George, and Robert E. McCulloch. “BART: Bayesian Additive Regression Trees.” The Annals of Applied Statistics 4.1 (2010): 266–298. Crossref. Web.