sparsevb 0.1.1

Spike-and-Slab Variational Bayes for Linear and Logistic Regression

sparsevb

A Python package for Spike-and-Slab Variational Bayes for Linear and Logistic Regression.

Description

This package provides variational Bayesian algorithms to perform scalable variable selection for sparse, high-dimensional linear and logistic regression models. The algorithms are implemented in C++ for performance and exposed to Python using Cython.

It is a direct port of the R sparsevb package, using the same underlying C++ logic but with a Pythonic API.

Installation

To install the package from source:

pip install .

Usage

Here is a basic example of how to use the package:

import numpy as np
from sparsevb import svb_fit_linear, svb_fit_logit

# Generate some synthetic data
np.random.seed(42)
n_samples, n_features = 200, 20
X = np.random.randn(n_samples, n_features)
beta_true = np.zeros(n_features)
beta_true[:3] = [1.5, -2.0, 3.0]
intercept_true = 5.0
y = X.dot(beta_true) + intercept_true + 0.1 * np.random.randn(n_samples)

# Fit a sparse linear regression model
res = svb_fit_linear(X, y, intercept=True)
print("Estimated coefficients (mu * gamma):", res['mu'] * res['gamma'])
print("Estimated intercept:", res['intercept'])
print("Estimated noise variance (noise_sd):", res['noise_sd'])

# For logistic regression (binary classification)
logits = X.dot(beta_true) + intercept_true
probs = 1 / (1 + np.exp(-logits))
y_binary = (np.random.rand(n_samples) < probs).astype(float)

res_logit = svb_fit_logit(X, y_binary, intercept=True)
print("Estimated logistic coefficients:", res_logit['mu'] * res_logit['gamma'])
print("Estimated logistic intercept:", res_logit['intercept'])

Features

• Prioritized Initialization: Uses Ridge and Lasso (via scikit-learn) to provide robust starting points.
• Intercept Handling: Built-in support for model intercepts.
• Noise Estimation: Automatic estimation of residual noise for linear models.
• Same Core Logic: Uses the exact same C++ algorithms as the original R package for reliable results.

Dependencies

• Python >= 3.8
• NumPy >= 1.20.0
• SciPy >= 1.6.0
• scikit-learn >= 1.0.0
• Cython >= 0.29.0

License

This project is licensed under the GNU General Public License v3.0.

References

• Kolyan Ray and Botond Szabo (2020). "Scalable Spike-and-Slab Variational Bayes for Linear and Logistic Regression." DOI:10.1080/01621459.2020.1847121