Comparing Ordinary Least Squares, Ridge, and Lasso Regression for Multicollinearity Mitigation in Linear Models

Abstract

Ordinary Least Squares (OLS) regression provides unbiased estimates but performs poorly when predictor variables are highly correlated, due to increased variance and model instability. This study compares the effectiveness of OLS, Ridge regression, and the Least Absolute Shrinkage and Selection Operator (LASSO) in mitigating multicollinearity and improving predictive accuracy in linear models. Using academic data of University of Ilorin undergraduate students, Nigeria, we evaluated model performance using Root Mean Squared Error (RMSE), Variance Inflation Factors (VIF), and cross-validation. Ridge regression applies an L₂ penalty to shrink coefficients, while LASSO uses an L₁ penalty that also enables variable selection by setting some coefficients to zero. The results show that Ridge regression achieved the best generalization performance with the lowest test RMSE (0.2358), while LASSO provided a more interpretable model through coefficient sparsity. OLS exhibited overfitting and the poorest generalization due to high multicollinearity. The findings highlight the importance of regularization techniques in regression modeling, especially in high-dimensional data environments. This study offers practical guidance on model selection when predictive accuracy and feature interpretability are essential.an