On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding

Authors:
Benedikt M. Pötscher;Hannes Leeb
Affiliations:
Department of Statistics, University of Vienna, Austria;Department of Statistics, Yale University, United States
Venue:
Journal of Multivariate Analysis
Year:
2009

Citing 1
Cited 3

On Model Selection Consistency of Lasso

The Journal of Machine Learning Research

Bootstrapping the likelihood ratio cointegration test in error correction models with unknown lag order

Computational Statistics & Data Analysis
Sparsity-aware estimation of CDMA system parameters

EURASIP Journal on Advances in Signal Processing - Special issue on advanced equalization techniques for wireless communications
Bootstrapping in non-regular smooth function models

Journal of Multivariate Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conservative model selection. Our findings complement those of Knight and Fu [K. Knight, W. Fu, Asymptotics for lasso-type estimators, Annals of Statistics 28 (2000) 1356-1378] and Fan and Li [J. Fan, R. Li, Variable selection via non-concave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96 (2001) 1348-1360]. We show that the distributions are typically highly non-normal regardless of how the estimator is tuned, and that this property persists in large samples. The uniform convergence rate of these estimators is also obtained, and is shown to be slower than n^-^1^/^2 in case the estimator is tuned to perform consistent model selection. An impossibility result regarding estimation of the estimators' distribution function is also provided.