SRLR 0.1.0

A package for sketched ridgeless estimator simulations, optimizing generalization. Identify the best sketching size to minimize out-of-sample risks. Stable risk curves in optimally sketched estimator eliminate peaks found in full-sample estimator. SRLR offers practical method to discover the ideal sketching size.

SRLR

Sketched Ridgeless Linear Regression

Description

This repository presents numerical simulations that analyze the empirical risks of the sketched ridgeless estimator, aiming to enhance generalization performance. The simulations focus on determining optimal sketching sizes that minimize out-of-sample prediction risks. The results reveal that the optimally sketched estimator exhibits stable risk curves, effectively eliminating the peaks observed in the full-sample estimator. Additionally, we introduce a practical procedure to empirically identify the optimal sketching size.

Suppose we observe data vectors (x_i,y_i) that follow a linear model y_i=x_i^Tβ+ε_i, i=1,...n, where y_i is a univariate response, x_i is a d-dimensional predictor, β denotes the vector of regression coefficients, and ε_i is a random error. We consider the ridgeless least square estimator β̂=(X^TX)⁺X^TY.

With this package, the simulation results in this paper can be reporduced.

Examples

Please refer to tutorial.ipynb for a comprehensive example and step-by-step guide.

Reference

Chen, X., Zeng, Y., Yang, S. and Sun, Q. Sketched Ridgeless Linear Regression: The Role of Downsampling. Paper