A scalable package for fitting sparse linear regression models with pairwise feature interactions, under strong hierarchy.
Project description
hierScale
Hussein Hazimeh and Rahul Mazumder
Massachusetts Institute of Technology
Introduction
hierScale is scalable toolkit for fitting sparse linear regression models with pairwise feature interactions. The optimization is done under the strong hierarchy (SH) constraint: an interaction coefficient is nonzero only if its associated main feature coefficients are nonzero. This constraint can enhance the interpretability of sparse interaction models and also reduce the future data collection costs; see the discussion in (Hazimeh and Mazumder, 2020).
More formally, given a data matrix X of main features and a response vector y, the toolkit fits a convex relaxation of the following model:
where X_i denotes the ith column (feature) of X and * refers to elementwise multiplication. The L0 norms impose sparsity on the coefficients and the constraints enforce SH. See (Hazimeh and Mazumder, 2020) for details on how the convex relaxation of the above problem is derived. The optimization is done for a regularization path (i.e., over a grid of lambda_1's and lambda_2's). We use proximal gradient descent (PGD) for optimization, along with novel proximal screening and gradient screening rules, which speed up PGD by over 4900x.
Installation
hierScale is written in Python 3. It uses Gurobi internally (for solving the LPs required for checking the optimality conditions). Before installing hierScale, please make sure that Gurobi and its Python interface (gurobipy) are installed.
To install hierScale, run the following command:
pip install hierScale
Quick Start
In Python, assuming you have the data X and y stored as numpy arrays, run the following to fit a regularization path:
from hierScale import hier_fit, hier_predict # Set the parameters of the algorithm. params = { "nLambda": 100, # Number of lambda_1's in the path. "maxSuppSize": 500, # Max support size to terminate the path at. } # Fit a path. solutions, lambdas = hier_fit(X, y, params) # solutions is a list of all the solutions in the path. # To access the ith solution, say i=10, use the following: current_solution = solutions[10] current_solution.B # A dictionary of the nonzero coefficients in beta. current_solution.T # A dictionary of the nonzero coefficients in theta. current_solution.intercept # The intercept term. # To predict the response given a matrix X, run the following: hier_predict(current_solution, X) # For more advanced usage and parameters, please check the documentation: print(hier_fit.__doc__)
References
Learning Hierarchical Interactions at Scale: A Convex Optimization Approach.
Bibtex citation below:
@inproceedings{hazimeh_mazumder_2020,
author = {Hussein Hazimeh and
Rahul Mazumder},
title = {Learning Hierarchical Interactions at Scale: A Convex Optimization Approach},
booktitle = {The 23nd International Conference on Artificial Intelligence and Statistics,
{AISTATS} 2020, 35 June 2020, Palermo, Sicily, Italy},
series = {Proceedings of Machine Learning Research},
volume = {108},
publisher = {{PMLR}},
year = {2020},
url = {https://arxiv.org/abs/1902.01542},
}
Project details
Release history Release notifications  RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Filename, size  File type  Python version  Upload date  Hashes 

Filename, size hierScale1.0.2py3noneany.whl (3.8 kB)  File type Wheel  Python version py3  Upload date  Hashes View 
Hashes for hierScale1.0.2py3noneany.whl
Algorithm  Hash digest  

SHA256  ad50274225b4b735c6e6443cad69b93b07998b9a1d821dce33bcac9630b54e11 

MD5  847fb8a8aebfae7bb5c0a7c802781aa3 

BLAKE2256  56a9e367bce325c60272cd0697603457b382ae8711afeabbd343334d8619bdcb 