EE Seminar: Selecting the LASSO Regularization Parameter via Bayesian Principles

Speaker: Naor Huri

M.Sc. student under the supervision of Prof. Meir Feder

Wednesday, September 21st 2016, at 15:00

Room 011, Kitot Bldg., Faculty of Engineering

Selecting the LASSO Regularization Parameter via Bayesian Principles

We consider the problem of model selection and coefficients estimation in linear regression. Our focus is LASSO regression, which has gained increasing popularity in recent years due to its ability to select sparse and simple solutions. We address the problem of selecting the LASSO regularization constant and introduce a new criterion for choosing it. This criterion is based on Bayesian inference principles, where models and regularizing constants are set by examining their posterior probability distribution and by evaluating the evidence for them. In particular, we propose a new geometrical interpretation of the LASSO solution that helps us to approximate the integrals involved in the calculation of the evidence, which are considered intractable. The resulting criterion can be computed at the cost of calculating matrix determinant, and therefore is a simple alternative to exhausting cross validation. We show that our proposed technique gives comparable and sometimes superior performance compared with other methods of similar complexity like the Generalized Cross Validation (GCV), the extended Akaike Information Criterion (AIC) and the extended Bayesian Information Criterion (BIC).