Bias-Variance Analysis and Ensembles of SVM

  • Authors:
  • Giorgio Valentini;Thomas G. Dietterich

  • Affiliations:
  • -;-

  • Venue:
  • MCS '02 Proceedings of the Third International Workshop on Multiple Classifier Systems
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

Accuracy, diversity, and learning characteristics of base learners critically influence the effectiveness of ensemble methods. Bias-variance decomposition of the error can be used as a tool to gain insights into the behavior of learning algorithms, in order to properly design ensemble methods well-tuned to the properties of a specific base learner. In this work we analyse bias-variance decomposition of the error in Support Vector Machines (SVM), characterizing it with respect to the kernel and its parameters. We show that the bias-variance decomposition offers a rationale to develop ensemble methods using SVMs as base learners, and we outline two directions for developing SVM ensembles, exploiting the SVM bias characteristics and the bias-variance dependence on the kernel parameters.