Tuning parameter selection in sparse regression modeling

Authors:
Kei Hirose;Shohei Tateishi;Sadanori Konishi
Affiliations:
Graduate School of Engineering Science, Osaka University, 1-3, Machikaneyama-cho, Toyonaka, Osaka 560-8531, Japan;Toyama Chemical Co., Ltd., 3-2-5, Nishi-Shinjuku, Shinjuku-ku, Tokyo 160-0023, Japan;Faculty of Science and Engineering, Chuo University, 1-13-27 Kasuga, Bunkyo-ku, Tokyo 112-8551, Japan
Venue:
Computational Statistics & Data Analysis
Year:
2013

Citing 4
Cited 1

On the degrees of freedom in shrinkage estimation

Journal of Multivariate Analysis
Information Criteria and Statistical Modeling

Information Criteria and Statistical Modeling
Slope heuristics: overview and implementation

Statistics and Computing
NNRMLR: A Combined Method of Nearest Neighbor Regression and Multiple Linear Regression

IIAI-AAI '12 Proceedings of the 2012 IIAI International Conference on Advanced Applied Informatics

Model selection in kernel ridge regression

Computational Statistics & Data Analysis

Quantified Score

Hi-index	0.03

Visualization

Abstract

In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows' C"p type criteria may be used as a tuning parameter selection tool in lasso type regularization methods, for which the concept of degrees of freedom plays a key role. In the present paper, we propose an efficient algorithm that computes the degrees of freedom by extending the generalized path seeking algorithm. Our procedure allows us to construct model selection criteria for evaluating models estimated by regularization with a wide variety of convex and nonconvex penalties. The proposed methodology is investigated through the analysis of real data and Monte Carlo simulations. Numerical results show that C"p criterion based on our algorithm performs well in various situations.