Reduction Techniques for Instance-BasedLearning Algorithms

  • Authors:
  • D. Randall Wilson;Tony R. Martinez

  • Affiliations:
  • Neural Network & Machine Learning Laboratory, Computer Science Department, Brigham Young University, Provo, Utah 84602, USA. randy@axon.cs.byu.edu;Neural Network & Machine Learning Laboratory, Computer Science Department, Brigham Young University, Provo, Utah 84602, USA. martinez@cs.byu.edu

  • Venue:
  • Machine Learning
  • Year:
  • 2000

Quantified Score

Hi-index 0.01

Visualization

Abstract

Instance-based learning algorithms are often faced with the problemof deciding which instances to store for use during generalization.Storing too many instances can result in large memory requirementsand slow execution speed, and can cause an oversensitivity to noise.This paper has two main purposes. First, it provides a survey ofexisting algorithms used to reduce storage requirements ininstance-based learning algorithms and other exemplar-basedalgorithms. Second, it proposes six additional reduction algorithmscalled DROP1–DROP5 and DEL (three of which werefirst described in Wilson & Martinez, 1997c, asRT1–RT3) that can be used to remove instances from the conceptdescription. These algorithms and 10 algorithms from the survey arecompared on 31 classification tasks. Of those algorithms thatprovide substantial storage reduction, the DROP algorithms havethe highest average generalization accuracy in these experiments,especially in the presence of uniform class noise.