Regularized nonsmooth Newton method for multi-class support vector machines

Authors:
Ping Zhong;Masao Fukushima
Affiliations:
College of Science, China Agricultural University, Beijing, China,Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto, Japan;Department of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto, Japan
Venue:
Optimization Methods & Software - Systems Analysis, Optimization and Data Mining in Biomedicine
Year:
2007

Citing 12
Cited 3

A nonmonotone line search technique for Newton's method

SIAM Journal on Numerical Analysis
Equivalent differentiable optimization problems and descent methods for asymmetric variational inequality problems

Mathematical Programming: Series A and B
Convergence analysis of some algorithms for solving nonsmooth equations

Mathematics of Operations Research
Unconstrained optimization reformulations of variational inequality problems

Journal of Optimization Theory and Applications
Equivalence of variational inequality problems to unconstrained minimization

Mathematical Programming: Series A and B
Pairwise classification and support vector machines

Advances in kernel methods
Combining support vector and mathematical programming methods for classification

Advances in kernel methods
Classification by pairwise coupling

NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Reducing multiclass to binary: a unifying approach for margin classifiers

The Journal of Machine Learning Research
On the algorithmic implementation of multiclass kernel-based vector machines

The Journal of Machine Learning Research
Solving multiclass learning problems via error-correcting output codes

Journal of Artificial Intelligence Research
Solving box constrained variational inequalities by using the natural residual with D-gap function globalization

Operations Research Letters

Spike-timing error backpropagation in theta neuron networks

Neural Computation
Smoothing Newton Method for L1 Soft Margin Data Classification Problem

ICCS 2009 Proceedings of the 9th International Conference on Computational Science
Let us know your decision: Pool-based active training of a generative classifier with the selection strategy 4DS

Information Sciences: an International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

Multi-class classification is an important and on-going research subject in machine learning. Recently, the ν-K-SVCR method was proposed by the authors for multi-class classification. As many optimization problems have to be solved in multi-class classification, it is extremely important to develop an algorithm that can solve those optimization problems efficiently. In this article, the optimization problem in the ν-K-SVCR method is reformulated as an affine box constrained variational inequality problem with a positive semi-definite matrix, and a regularized version of the nonsmooth Newton method that uses the D-gap function as a merit function is applied to solve the resulting problems. The proposed algorithm fully exploits the typical feature of the ν-K-SVCR method, which enables us to reduce the size of Newton equations significantly. This indicates that the algorithm can be implemented efficiently in practice. The preliminary numerical experiments on benchmark data sets show that the proposed method is considerably faster than the standard Matlab routine used in the original ν-K-SVCR method.