A method for large-scale l1-regularized logistic regression

Authors:
Kwangmoo Koh;Seung-Jean Kim;Stephen Boyd
Affiliations:
Electrical Engineering Department, Stanford University, Stanford, CA;Electrical Engineering Department, Stanford University, Stanford, CA;Electrical Engineering Department, Stanford University, Stanford, CA
Venue:
AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 1
Year:
2007

Citing 8
Cited 1

Minimization methods for non-differentiable functions

Minimization methods for non-differentiable functions
Applied numerical linear algebra

Applied numerical linear algebra
Convex Optimization

Convex Optimization
Feature selection, L1 vs. L2 regularization, and rotational invariance

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Sparse Multinomial Logistic Regression: Fast Algorithms and Generalization Bounds

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs

The Journal of Machine Learning Research
Gene selection in cancer classification using sparse logistic regression with Bayesian regularization

Bioinformatics
The generalized LASSO

IEEE Transactions on Neural Networks

Functional site prediction by exploiting correlations between labels of interacting residues

Proceedings of the ACM Conference on Bioinformatics, Computational Biology and Biomedicine

Quantified Score

Hi-index	0.00

Visualization

Abstract

Logistic regression with l1 regularization has been proposed as a promising method for feature selection in classification problems. Several specialized solution methods have been proposed for l1-regularized logistic regression problems (LRPs). However, existing methods do not scale well to large problems that arise in many practical settings. In this paper we describe an efficient interior-point method for solving l1-regularized LRPS. Small problems with up to a thousand or so features and examples can be solved in seconds on a PC. A variation on the basic method, that uses a preconditioned conjugate gradient method to compute the search step, can solve large sparse problems, with a million features and examples (e.g., the 20 Newsgroups data set), in a few tens of minutes, on a PC. Numerical experiments show that our method outperforms standard methods for solving convex optimization problems as well as other methods specifically designed for l1- regularized LRPs.