An improved algorithm for the solution of the regularization path of support vector machine

Authors:
Chong-Jin Ong;Shiyun Shao;Jianbo Yang
Affiliations:
Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore and Singapore-MIT Alliance, Singapore, Singapore;Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore;Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore
Venue:
IEEE Transactions on Neural Networks
Year:
2010

Citing 14
Cited 1

Making large-scale support vector machine learning practical

Advances in kernel methods
Fast training of support vector machines using sequential minimal optimization

Advances in kernel methods
Linear Programming in Linear Time When the Dimension Is Fixed

Journal of the ACM (JACM)
Alpha seeding for support vector machines

Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining
The Entire Regularization Path for the Support Vector Machine

The Journal of Machine Learning Research
A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs

The Journal of Machine Learning Research
Improvements to Platt's SMO Algorithm for SVM Classifier Design

Neural Computation
Working Set Selection Using Second Order Information for Training Support Vector Machines

The Journal of Machine Learning Research
Efficient Computation and Model Selection for the Support Vector Regression

Neural Computation
A kernel path algorithm for support vector machines

Proceedings of the 24th international conference on Machine learning
A dual coordinate descent method for large-scale linear SVM

Proceedings of the 25th international conference on Machine learning
Exponentiated Gradient Algorithms for Conditional Random Fields and Max-Margin Markov Networks

The Journal of Machine Learning Research
An efficient method for computing leave-one-out error in support vector machines with Gaussian kernels

IEEE Transactions on Neural Networks
A New Solution Path Algorithm in Support Vector Regression

IEEE Transactions on Neural Networks

On the SVMpath initialization

Signal Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes an improved algorithm for the numerical solution to the support vector machine (SVM) classification problem for all values of the regularization parameter C. The algorithm is motivated by the work of Hastie et al. and follows the main idea of tracking the optimality conditions of the SVM solution for ascending value of C. It differs from Hastie's approach in that the tracked path is not assumed to be 1-D. Instead, a multidimensional feasible space for the optimality condition is used to solve the tracking problem. Such a treatment allows the algorithm to properly handle data sets which Hastie's approach fails. These data sets are characterized by the presence of linearly dependent points (in the kernel space), duplicate points, or nearly duplicate points. Such data sets are quite common among many real-world data, especially those with nominal features. Other contributions of this paper include a unifying formulation of the tracking process in the form of a linear programming problem, update formula for the linear programs, considerations that guard against accumulation of errors resulting from the use of incremental updates, and routines to speed up the algorithm. The algorithm is implemented under the Matlab environment and is available for download. Experiments with several data sets including data set having up to several thousand data points are reported.