Large Margin Classification Using the Perceptron Algorithm

  • Authors:
  • Yoav Freund;Robert E. Schapire

  • Affiliations:
  • AT&T Labs, Shannon Laboratory, 180 Park Avenue, Room A205, Florham Park, NJ 07932-0971, USA. yoav@research.att.com;AT&T Labs, Shannon Laboratory, 180 Park Avenue, Room A279, Florham Park, NJ 07932-0971, USA. schapire@research.att.com

  • Venue:
  • Machine Learning - The Eleventh Annual Conference on computational Learning Theory
  • Year:
  • 1999

Quantified Score

Hi-index 0.00

Visualization

Abstract

We introduce and analyze a new algorithm for linear classification which combines Rosenblatt‘s perceptron algorithm with Helmbold and Warmuth‘s leave-one-out method. Like Vapnik‘s maximal-margin classifier, our algorithm takesadvantage of data that are linearly separable with large margins.Compared to Vapnik‘s algorithm, however,ours is much simpler to implement, andmuch more efficient in terms of computation time.We also show that our algorithm can be efficiently used in very highdimensional spaces using kernel functions.We performed some experiments using our algorithm, and some variantsof it, for classifying images of handwritten digits.The performance of our algorithm is close to, but not as good as, theperformance of maximal-margin classifiers on the same problem, whilesaving significantly on computation time and programming effort.