The Racing Algorithm: Model Selection for Lazy Learners

  • Authors:
  • Oden Maron;Andrew W. Moore

  • Affiliations:
  • M.I.T. Artificial Intelligence Lab, NE45-755, 545 Technology Square, Cambridge, MA 02139. E-mail: oded@ai.mit.edu;Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA 15213. E-mail: awm@cs.cmu.edu

  • Venue:
  • Artificial Intelligence Review - Special issue on lazy learning
  • Year:
  • 1997

Quantified Score

Hi-index 0.00

Visualization

Abstract

Given a set of models and some training data, we would like to findthe model that best describes the data. Finding the model with thelowest generalization error is a computationally expensive process,especially if the number of testing points is high or if the number ofmodels is large. Optimization techniques such as hill climbing orgenetic algorithms are helpful but can end up with a model that isarbitrarily worse than the best one or cannot be usedbecause there is no distance metric on the space of discrete models.In this paper we develop a technique called ’’racing‘‘ that tests theset of models in parallel, quickly discards those models that areclearly inferior and concentrates the computational effort ondifferentiating among the better models. Racing is especiallysuitable for selecting among lazy learners since training requiresnegligible expense, and incremental testing using leave-one-out crossvalidation is efficient. We use racing to select among various lazylearning algorithms and to find relevant features in applicationsranging from robot juggling to lesion detection in MRI scans.