Efficient methods for estimating constrained parameters with applications to regularized (lasso) logistic regression

Authors:
Guo-Liang Tian;Man-Lai Tang;Hong-Bin Fang;Ming Tan
Affiliations:
Division of Biostatistics, University of Maryland Greenebaum Cancer Center, 10 South Pine Street, MSTF Suite 261, Baltimore, MD 21201, USA;Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong, China;Division of Biostatistics, University of Maryland Greenebaum Cancer Center, 10 South Pine Street, MSTF Suite 261, Baltimore, MD 21201, USA;Division of Biostatistics, University of Maryland Greenebaum Cancer Center, 10 South Pine Street, MSTF Suite 261, Baltimore, MD 21201, USA
Venue:
Computational Statistics & Data Analysis
Year:
2008

Citing 2
Cited 2

Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization

ACM Transactions on Mathematical Software (TOMS)
Multiclass sparse logistic regression for classification of multiple cancer types using gene expression data

Computational Statistics & Data Analysis

Favorability functions based on kernel density estimation for logistic models: A case study

Computational Statistics & Data Analysis
Automated classification of dopaminergic neurons in the rodent brain

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks

Quantified Score

Hi-index	0.03

Visualization

Abstract

Fitting logistic regression models is challenging when their parameters are restricted. In this article, we first develop a quadratic lower-bound (QLB) algorithm for optimization with box or linear inequality constraints and derive the fastest QLB algorithm corresponding to the smallest global majorization matrix. The proposed QLB algorithm is particularly suited to problems to which the EM-type algorithms are not applicable (e.g., logistic, multinomial logistic, and Cox's proportional hazards models) while it retains the same EM ascent property and thus assures the monotonic convergence. Secondly, we generalize the QLB algorithm to penalized problems in which the penalty functions may not be totally differentiable. The proposed method thus provides an alternative algorithm for estimation in lasso logistic regression, where the convergence of the existing lasso algorithm is not generally ensured. Finally, by relaxing the ascent requirement, convergence speed can be further accelerated. We introduce a pseudo-Newton method that retains the simplicity of the QLB algorithm and the fast convergence of the Newton method. Theoretical justification and numerical examples show that the pseudo-Newton method is up to 71 (in terms of CPU time) or 107 (in terms of number of iterations) times faster than the fastest QLB algorithm and thus makes bootstrap variance estimation feasible. Simulations and comparisons are performed and three real examples (Down syndrome data, kyphosis data, and colon microarray data) are analyzed to illustrate the proposed methods.