Skip to main content

A scalable package for fitting sparse linear regression models with pairwise feature interactions, under strong hierarchy.

Project description

hierScale

Hussein Hazimeh and Rahul Mazumder

Massachusetts Institute of Technology

Introduction

hierScale is scalable toolkit for fitting sparse linear regression models with pairwise feature interactions. The optimization is done under the strong hierarchy (SH) constraint: an interaction coefficient is non-zero only if its associated main feature coefficients are non-zero. This constraint can enhance the interpretability of sparse interaction models and also reduce the future data collection costs; see the discussion in (Hazimeh and Mazumder, 2020).

More formally, given a data matrix X of main features and a response vector y, the toolkit fits a convex relaxation of the following model:

where X_i denotes the ith column (feature) of X and * refers to element-wise multiplication. The L0 norms impose sparsity on the coefficients and the constraints enforce SH. See (Hazimeh and Mazumder, 2020) for details on how the convex relaxation of the above problem is derived. The optimization is done for a regularization path (i.e., over a grid of lambda_1's and lambda_2's). We use proximal gradient descent (PGD) for optimization, along with novel proximal screening and gradient screening rules, which speed up PGD by over 4900x.

Installation

hierScale is written in Python 3. It uses Gurobi internally (for solving the LPs required for checking the optimality conditions). Before installing hierScale, please make sure that Gurobi and its Python interface (gurobipy) are installed.

To install hierScale, run the following command:

pip install hierScale

Quick Start

In Python, assuming you have the data X and y stored as numpy arrays, run the following to fit a regularization path:

from hierScale import hier_fit, hier_predict

# Set the parameters of the algorithm.
params = {
    "nLambda": 100, # Number of lambda_1's in the path.
    "maxSuppSize": 500, # Max support size to terminate the path at.
}

# Fit a path.
solutions, lambdas = hier_fit(X, y, params)

# solutions is a list of all the solutions in the path.
# To access the ith solution, say i=10, use the following:
current_solution = solutions[10]
current_solution.B # A dictionary of the non-zero coefficients in beta.
current_solution.T # A dictionary of the non-zero coefficients in theta.
current_solution.intercept # The intercept term.

# To predict the response given a matrix X, run the following:
hier_predict(current_solution, X)

# For more advanced usage and parameters, please check the documentation:
print(hier_fit.__doc__)

References

Learning Hierarchical Interactions at Scale: A Convex Optimization Approach.

Bibtex citation below:

@InProceedings{pmlr-v108-hazimeh20a, 
title = {Learning Hierarchical Interactions at Scale: A Convex Optimization Approach},
author = {Hazimeh, Hussein and Mazumder, Rahul},
booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics},
pages = {1833--1843},
year = {2020},
editor = {Silvia Chiappa and Roberto Calandra},
volume = {108},
series = {Proceedings of Machine Learning Research},
address = {Online},
month = {26--28 Aug},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v108/hazimeh20a/hazimeh20a.pdf},
url = {http://proceedings.mlr.press/v108/hazimeh20a.html}}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hierScale-1.0.6.tar.gz (19.5 kB view details)

Uploaded Source

File details

Details for the file hierScale-1.0.6.tar.gz.

File metadata

  • Download URL: hierScale-1.0.6.tar.gz
  • Upload date:
  • Size: 19.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.0 CPython/3.9.7

File hashes

Hashes for hierScale-1.0.6.tar.gz
Algorithm Hash digest
SHA256 47a0f1963c6d5d3aa50c9a98f4191d657804e2cd6bb3b7200d3fd6eab0bf6140
MD5 ac9abbc6d4176d5e8bd3d4d6ebecb693
BLAKE2b-256 d9b98eab52aa2cee440f4b8e418cf27021876b785bc5b57e0169956f85040a2a

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page