Multiclass LS-SVMs: Moderated Outputs and Coding-Decoding Schemes

  • Authors:
  • T. Van Gestel;J. A. K. Suykens;G. Lanckriet;A. Lambrechts;B. De Moor;J. Vandewalle

  • Affiliations:
  • Katholieke Universiteit Leuven, Dept. of Electrical Engineering – ESAT/SISTA, Kard. Mercierlaan 94, B-3001 Leuven, Belgium. E-mail: tony.vangestel@esat.kuleuven.ac.be;Katholieke Universiteit Leuven, Dept. of Electrical Engineering – ESAT/SISTA, Kard. Mercierlaan 94, B-3001 Leuven, Belgium. E-mail: johan.suykens@esat.kuleuven.ac.be;Katholieke Universiteit Leuven, Dept. of Electrical Engineering – ESAT/SISTA, Kard. Mercierlaan 94, B-3001 Leuven, Belgium;Katholieke Universiteit Leuven, Dept. of Electrical Engineering – ESAT/SISTA, Kard. Mercierlaan 94, B-3001 Leuven, Belgium;Katholieke Universiteit Leuven, Dept. of Electrical Engineering – ESAT/SISTA, Kard. Mercierlaan 94, B-3001 Leuven, Belgium;Katholieke Universiteit Leuven, Dept. of Electrical Engineering – ESAT/SISTA, Kard. Mercierlaan 94, B-3001 Leuven, Belgium

  • Venue:
  • Neural Processing Letters
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

A common way of solving the multiclass categorization problem is to reformulate the problem into a set of binary classification problems. Discriminative binary classifiers like, e.g., Support Vector Machines (SVMs), directly optimize the decision boundary with respect to a certain cost function. In a pragmatic and computationally simple approach, Least Squares SVMs (LS-SVMs) are inferred by minimizing a related regression least squares cost function. The moderated outputs of the binary classifiers are obtained in a second step within the evidence framework. In this paper, Bayes' rule is repeatedly applied to infer the posterior multiclass probabilities, using the moderated outputs of the binary plug-in classifiers and the prior multiclass probabilities. This Bayesian decoding motivates the use of loss function based decoding instead of Hamming decoding. For SVMs and LS-SVMs with linear kernel, experimental evidence suggests the use of one-versus-one coding. With a Radial Basis Function kernel one-versus-one and error correcting output codes yield the best performances, but simpler codings may still yield satisfactory results.