Solving multiclass support vector machines with LaRank

Authors:
Antoine Bordes;Léon Bottou;Patrick Gallinari;Jason Weston
Affiliations:
NEC Laboratories America, Inc., Princeton, NJ Université de Paris, Paris, France;NEC Laboratories America, Inc., Princeton, NJ;Université de Paris, Paris, France;NEC Laboratories America, Inc., Princeton, NJ
Venue:
Proceedings of the 24th international conference on Machine learning
Year:
2007

Citing 12
Cited 24

Fast training of support vector machines using sequential minimal optimization

Advances in kernel methods
On the algorithmic implementation of multiclass kernel-based vector machines

The Journal of Machine Learning Research
Ultraconservative online algorithms for multiclass problems

The Journal of Machine Learning Research
In Defense of One-Vs-All Classification

The Journal of Machine Learning Research
Learning structured prediction models: a large margin approach

Learning structured prediction models: a large margin approach
Large Margin Methods for Structured and Interdependent Output Variables

The Journal of Machine Learning Research
Learning structured prediction models: a large margin approach

ICML '05 Proceedings of the 22nd international conference on Machine learning
Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms

EMNLP '02 Proceedings of the ACL-02 conference on Empirical methods in natural language processing - Volume 10
Fast Kernel Classifiers with Online and Active Learning

The Journal of Machine Learning Research
Predicting Structured Data (Neural Information Processing)

Predicting Structured Data (Neural Information Processing)
The huller: a simple and efficient online SVM

ECML'05 Proceedings of the 16th European conference on Machine Learning
A comparison of methods for multiclass support vector machines

IEEE Transactions on Neural Networks

A dual coordinate descent method for large-scale linear SVM

Proceedings of the 25th international conference on Machine learning
Accurate max-margin training for structured output spaces

Proceedings of the 25th international conference on Machine learning
Structured learning for non-smooth ranking losses

Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
A sequential dual method for large scale multi-class linear svms

Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
Sequence Labelling SVMs Trained in One Pass

ECML PKDD '08 Proceedings of the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases - Part I
Information Extraction

Foundations and Trends in Databases
Ranking with ordered weighted pairwise classification

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Automating knowledge capture in the aerospace domain

Proceedings of the fifth international conference on Knowledge capture
Multi-class classifiers and their underlying shared structure

IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Periodic step-size adaptation in second-order gradient descent for single-pass on-line structured learning

Machine Learning
Hybrid MPI/OpenMP Parallel Linear Support Vector Machine Training

The Journal of Machine Learning Research
Optimized Cutting Plane Algorithm for Large-Scale Risk Minimization

The Journal of Machine Learning Research
A Quasi-Newton Approach to Nonsmooth Convex Optimization Problems in Machine Learning

The Journal of Machine Learning Research
An online core vector machine with adaptive MEB adjustment

Pattern Recognition
How about utilizing ordinal information from the distribution of unlabeled data

CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Tree Decomposition for Large-Scale SVM Problems

The Journal of Machine Learning Research
Collective Inference for Extraction MRFs Coupled with Symmetric Clique Potentials

The Journal of Machine Learning Research
Double Updating Online Learning

The Journal of Machine Learning Research
Improved working set selection for larank

CAIP'11 Proceedings of the 14th international conference on Computer analysis of images and patterns - Volume Part I
Multiplicative update rules for incremental training of multiclass support vector machines

Pattern Recognition
Editors Choice Article: I2VM: Incremental import vector machines

Image and Vision Computing
Distribution-aware online classifiers

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Two
A review of optimization methodologies in support vector machines

Neurocomputing
Robust tracking with weighted online structured learning

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part III

Quantified Score

Hi-index	0.01

Visualization

Abstract

Optimization algorithms for large margin multiclass recognizers are often too costly to handle ambitious problems with structured outputs and exponential numbers of classes. Optimization algorithms that rely on the full gradient are not effective because, unlike the solution, the gradient is not sparse and is very large. The LaRank algorithm sidesteps this difficulty by relying on a randomized exploration inspired by the perceptron algorithm. We show that this approach is competitive with gradient based optimizers on simple multiclass problems. Furthermore, a single LaRank pass over the training examples delivers test error rates that are nearly as good as those of the final solution.