An empirical study of Hoeffding racing for model selection in k-nearest neighbor classification

Authors:
Flora Yu-Hui Yeh;Marcus Gallagher
Affiliations:
School of Information Technology and Electrical Engineering, University of Queensland, Australia;School of Information Technology and Electrical Engineering, University of Queensland, Australia
Venue:
IDEAL'05 Proceedings of the 6th international conference on Intelligent Data Engineering and Automated Learning
Year:
2005

Citing 4
Cited 0

Locally Weighted Learning

Artificial Intelligence Review - Special issue on lazy learning
The Racing Algorithm: Model Selection for Lazy Learners

Artificial Intelligence Review - Special issue on lazy learning
Machine Learning

Machine Learning
A Racing Algorithm for Configuring Metaheuristics

GECCO '02 Proceedings of the Genetic and Evolutionary Computation Conference

Quantified Score

Hi-index	0.00

Visualization

Abstract

Racing algorithms have recently been proposed as a general-purpose method for performing model selection in machine learning algorithms. In this paper, we present an empirical study of the Hoeffding racing algorithm for selecting the k parameter in a simple k-nearest neighbor classifier. Fifteen widely-used classification datasets from UCI are used and experiments conducted across different confidence levels for racing. The results reveal a significant amount of sensitivity of thek -nn classifier to its model parameter value. The Hoeffding racing algorithm also varies widely in its performance, in terms of the computational savings gained over an exhaustive evaluation. While in some cases the savings gained are quite small, the racing algorithm proved to be highly robust to the possibility of erroneously eliminating the optimal models. All results were strongly dependent on the datasets used.