Minimization methods for non-differentiable functions
Minimization methods for non-differentiable functions
Uncertainty principles and signal recovery
SIAM Journal on Applied Mathematics
Signal recovery and the large sieve
SIAM Journal on Applied Mathematics
Nonlinear total variation based noise removal algorithms
Proceedings of the eleventh annual international conference of the Center for Nonlinear Studies on Experimental mathematics : computational issues in nonlinear science: computational issues in nonlinear science
Sparse Approximate Solutions to Linear Systems
SIAM Journal on Computing
Optimal wire and transistor sizing for circuits with non-tree topology
ICCAD '97 Proceedings of the 1997 IEEE/ACM international conference on Computer-aided design
Atomic Decomposition by Basis Pursuit
SIAM Journal on Scientific Computing
On Feature Selection: Learning with Exponentially Many Irrelevant Features as Training Examples
ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Adaptive Sparseness for Supervised Learning
IEEE Transactions on Pattern Analysis and Machine Intelligence
RCV1: A New Benchmark Collection for Text Categorization Research
The Journal of Machine Learning Research
Feature selection, L1 vs. L2 regularization, and rotational invariance
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Sparse Multinomial Logistic Regression: Fast Algorithms and Generalization Bounds
IEEE Transactions on Pattern Analysis and Machine Intelligence
Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics)
On Model Selection Consistency of Lasso
The Journal of Machine Learning Research
An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression
The Journal of Machine Learning Research
EfficientL1regularized logistic regression
AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Fixed-Point Continuation for $\ell_1$-Minimization: Methodology and Convergence
SIAM Journal on Optimization
Bregman Iterative Algorithms for $\ell_1$-Minimization with Applications to Compressed Sensing
SIAM Journal on Imaging Sciences
A Nonlinear Inverse Scale Space Method for a Convex Multiplicative Noise Model
SIAM Journal on Imaging Sciences
Uncertainty principles and ideal atomic decomposition
IEEE Transactions on Information Theory
IEEE Transactions on Information Theory
IEEE Transactions on Information Theory
Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?
IEEE Transactions on Information Theory
Optimizing dominant time constant in RC circuits
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
IEEE Transactions on Neural Networks
The Journal of Machine Learning Research
Accelerated Block-coordinate Relaxation for Regularized Optimization
SIAM Journal on Optimization
An improved GLMNET for L1-regularized logistic regression
The Journal of Machine Learning Research
Sublinear algorithms for penalized logistic regression in massive datasets
ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I
Hi-index | 0.00 |
l1-regularized logistic regression, also known as sparse logistic regression, is widely used in machine learning, computer vision, data mining, bioinformatics and neural signal processing. The use of l1 regularization attributes attractive properties to the classifier, such as feature selection, robustness to noise, and as a result, classifier generality in the context of supervised learning. When a sparse logistic regression problem has large-scale data in high dimensions, it is computationally expensive to minimize the non-differentiable l1-norm in the objective function. Motivated by recent work (Koh et al., 2007; Hale et al., 2008), we propose a novel hybrid algorithm based on combining two types of optimization iterations: one being very fast and memory friendly while the other being slower but more accurate. Called hybrid iterative shrinkage (HIS), the resulting algorithm is comprised of a fixed point continuation phase and an interior point phase. The first phase is based completely on memory efficient operations such as matrix-vector multiplications, while the second phase is based on a truncated Newton's method. Furthermore, we show that various optimization techniques, including line search and continuation, can significantly accelerate convergence. The algorithm has global convergence at a geometric rate (a Q-linear rate in optimization terminology). We present a numerical comparison with several existing algorithms, including an analysis using benchmark data from the UCI machine learning repository, and show our algorithm is the most computationally efficient without loss of accuracy.