Finding the upper envelope of n line segments in O(n log n) time
Information Processing Letters
On the limited memory BFGS method for large scale optimization
Mathematical Programming: Series A and B
Globally convergent variable metric method for convex nonsmooth unconstrained minimization
Journal of Optimization Theory and Applications
SSVM: A Smooth Support Vector Machine for Classification
Computational Optimization and Applications
Text Categorization Based on Regularized Linear Classification Methods
Information Retrieval
Kinetic data structures
Ultraconservative online algorithms for multiclass problems
The Journal of Machine Learning Research
A family of additive online algorithms for category ranking
The Journal of Machine Learning Research
Convex Optimization
Smooth minimization of non-smooth functions
Mathematical Programming: Series A and B
Large Margin Methods for Structured and Interdependent Output Variables
The Journal of Machine Learning Research
Training linear SVMs in linear time
Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
Scalable training of L1-regularized log-linear models
Proceedings of the 24th international conference on Machine learning
Solving multiclass support vector machines with LaRank
Proceedings of the 24th international conference on Machine learning
Optimized cutting plane algorithm for support vector machines
Proceedings of the 25th international conference on Machine learning
A quasi-Newton approach to non-smooth convex optimization
Proceedings of the 25th international conference on Machine learning
Optimized Cutting Plane Algorithm for Large-Scale Risk Minimization
The Journal of Machine Learning Research
Bundle Methods for Regularized Risk Minimization
The Journal of Machine Learning Research
The Journal of Machine Learning Research
Super-Linear Convergence of Dual Augmented Lagrangian Algorithm for Sparsity Regularized Estimation
The Journal of Machine Learning Research
Training Lp norm multiple kernel learning in the primal
Neural Networks
| Hi-index | 0.00 |
We extend the well-known BFGS quasi-Newton method and its memory-limited variant LBFGS to the optimization of nonsmooth convex objectives. This is done in a rigorous fashion by generalizing three components of BFGS to subdifferentials: the local quadratic model, the identification of a descent direction, and the Wolfe line search conditions. We prove that under some technical conditions, the resulting subBFGS algorithm is globally convergent in objective function value. We apply its memory-limited variant (subLBFGS) to L2-regularized risk minimization with the binary hinge loss. To extend our algorithm to the multiclass and multilabel settings, we develop a new, efficient, exact line search algorithm. We prove its worst-case time complexity bounds, and show that our line search can also be used to extend a recently developed bundle method to the multiclass and multilabel settings. We also apply the direction-finding component of our algorithm to L1-regularized risk minimization with logistic loss. In all these contexts our methods perform comparable to or better than specialized state-of-the-art solvers on a number of publicly available data sets. An open source implementation of our algorithms is freely available.