frenezik 0.1.3

Ridge and Lasso in Python for regression analysis, prediction and variables selection.

---

Description

Frenezik is a Python package that enables the development of Ridge and Lasso regularized regression models. These models can be designed using two approaches. The first approach is known as the simulation approach because it allows for the generation of data from the population and/or sample under study. This approach is suitable for researchers and students whose goal is to gain a deep understanding of Ridge and Lasso models. The second approach uses existing data, that is, non-simulated data.

Application

This package provides:

Advanced regression analyses that include estimates of regression coefficients regularized using the LSQR optimization algorithm, significance tests for the coefficients and residual analysis using 3D visualizations for Ridge. Variable selection methods based on the notion of shrinkage of penalized coefficients as a function of the penalty parameter alpha. Predictions based on machine learning. We can therefore evaluate regularized models by splitting the data into training and test sets. Frenezik is an open-source project that uses and is compatible with the following stacks : NumPy, SciPy, Matplotlib, Seaborn, Pandas, Scikit-Learn, and Statsmodels.

Tutorial

Installation frenezik

pip install frenezik

Import frenezik

import frenezik as fk

Simulation and visualization Residuals

# Population size
PopulationSize = 500

# Random seed
RandomSeed = 1980

# Instance of <<Residuals>> class
Resid = fk.Residuals(population_size=PopulationSize, random_seed=RandomSeed)

# Parameter of Mean
LawMean = 2.81

# Parameter of Variance
LawVar = 3.89

# Simulation of residuals
Residus = Resid.Simulation(law="normal", law_parameters=[LawMean, LawVar])

# Histogram of residuals simulated
Resid.Visualization(residuals_simulated=Residus, figure='histogram')

Generation X and y data

# True coefficients 
TrueCoefficients = np.arange(-20, 20, 4.5)

# Generation of X and y data with simulated residuals
TarPred = fk.TargetPredictors(population_size=PopulationSize, true_coefficients=TrueCoefficients)
data_population = TarPred.Simulation(residuals_simulated=Residus)
data_population.head(5)

# Lineplot of generated X and y data
plt.figure(figsize=(8,6))
TarPred.Visualization_X_y(data_simulated=data_population, figure='lineplot')

Ridge Regression with ($\alpha = 5$)

# penality factor (alpha)
AlphaValue = 5

# Intercept State
InterceptState = True

# Instance of <<Regularization>> 
Regul = fk.Regularization(predictors=Predictors,
                       target=Target,
                       alpha=AlphaValue,
                       intercept=InterceptState,
                       random_seed=RandomSeed)

# Results of Ridge regression
ResultsRidge = Regul.Penalized_Regression(model='ridge')
ResultsRidge

Notes : The result of Regularization.Penalized_Regression function when alpha is scalar and model is Lasso is not yet available.

Lasso Regularization Path with Lasso instead Ridge fore more shrinkage ($\alpha_{k}$)

# Range of penality factor (alpha)
AlphaList = np.linspace(0, 9, 19)

# Intercept
InterceptState = True

# Instance of <<regularization>>
Regul = fk.Regularization(predictors=Predictors,
                       target=Target,
                       alpha=AlphaList,
                       intercept=InterceptState,
                       random_seed=RandomSeed)

# Summary of penalized coefficients in function of alpha
SummaryCoefLasso = Regul.Penalized_Regression(model='lasso')
SummaryCoefLasso

# Regularization path of Lasso model
Regul.Visualization_Shrinking(penalized_coefficients_table=SummaryCoefLasso,
                              variables_selected=None, 
                              alpha_value_selected=None,<img width="582" height="459" alt="download" src="https://github.com/user-attachments/assets/1928380b-f4f4-4df5-bba5-c73b66999673" />
                              figure='curve')

Residuals Analysis

# Contour plot of residuals sum square loss
Beta1 = np.linspace(-25, 25, 50)
Beta2 = np.linspace(-25, 25, 50)
Regul.Visualization_Residuals(list_beta_1=Beta1,
                        list_beta_2=Beta2,
                        predictors=Predictors,
                        target=Target,
                        penalized_coefficients_table=SummaryCoefLasso,
                        var_1='X_2',
                        var_2='X_3',
                        intercept=0,
                        figure='contour_map')

Bibliography

• Hastie, T. (2009). The elements of statistical learning: data mining, inference, and prediction.
• Cule, E., Vineis, P., & De Iorio, M. (2011). Significance testing in ridge regression for genetic data. BMC bioinformatics, 12(1), 372.
• van Wieringen, W. N. (2015). Lecture notes on ridge regression. arXiv preprint arXiv:1509.09169.

Author

If you noticed some issues or you just want to improve the project, it's will be a pleasure for me to discuss with you ☎ :

• Frenezik is developped and maintained by : Bengone Akou Lajoie
• Github : beng-one | Email : [e-mail]
• ATLAS Organization