Voting over Multiple Condensed Nearest Neighbors

  • Authors:
  • Ethem Alpaydin

  • Affiliations:
  • Department of Computer Engineering, Boğaziçi University, TR-80815 Istanbul

  • Venue:
  • Artificial Intelligence Review - Special issue on lazy learning
  • Year:
  • 1997

Quantified Score

Hi-index 0.00

Visualization

Abstract

Lazy learning methods like the k-nearest neighborclassifier require storing the whole training set and may be toocostly when this set is large. The condensed nearest neighborclassifier incrementally stores a subset of the sample, thusdecreasing storage and computation requirements. We propose totrain multiple such subsets and take a vote over them, thuscombining predictions from a set of concept descriptions. Weinvestigate two voting schemes: simple voting where voters haveequal weight and weighted voting where weights depend onclassifiers‘ confidences in their predictions. We consider waysto form such subsets for improved performance: When the trainingset is small, voting improves performance considerably. If thetraining set is not small, then voters converge to similarsolutions and we do not gain anything by voting. To alleviatethis, when the training set is of intermediate size, we usebootstrapping to generate smaller training sets over which wetrain the voters. When the training set is large, we partition itinto smaller, mutually exclusive subsets and then train thevoters. Simulation results on six datasets are reported with goodresults. We give a review of methods for combining multiplelearners. The idea of taking a vote over multiple learners can beapplied with any type of learning scheme.