Polynomial-Time Decomposition Algorithms for Support Vector Machines

  • Authors:
  • Don Hush;Clint Scovel

  • Affiliations:
  • Los Alamos National Laboratory, Los Alamos, NM, 87545, USA. dhush@lanl.gov;Los Alamos National Laboratory, Los Alamos, NM, 87545, USA. jcs@lanl.gov

  • Venue:
  • Machine Learning
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper studies the convergence properties of a general class of decomposition algorithms for support vector machines (SVMs). We provide a model algorithm for decomposition, and prove necessary and sufficient conditions for stepwise improvement of this algorithm. We introduce a simple “rate certifying” condition and prove a polynomial-time bound on the rate of convergence of the model algorithm when it satisfies this condition. Although it is not clear that existing SVM algorithms satisfy this condition, we provide a version of the model algorithm that does. For this algorithm we show that when the slack multiplier C satisfies \sqrt{1/2} ≤ C ≤ mL, where m is the number of samples and L is a matrix norm, then it takes no more than 4LC2m4/ε iterations to drive the criterion to within ε of its optimum.