Neighborhood size selection in the k-nearest-neighbor rule using statistical confidence

Authors:
Jigang Wang;Predrag Neskovic;Leon N. Cooper
Affiliations:
Department of Physics, The Institute for Brain and Neural Systems, P.O. Box 1843, Brown University, Providence, RI 02912, USA;Department of Physics, The Institute for Brain and Neural Systems, P.O. Box 1843, Brown University, Providence, RI 02912, USA;Department of Physics, The Institute for Brain and Neural Systems, P.O. Box 1843, Brown University, Providence, RI 02912, USA
Venue:
Pattern Recognition
Year:
2006

Citing 4
Cited 9

Neural networks and the bias/variance dilemma

Neural Computation
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
On the Inequality of Cover and Hart in Nearest Neighbor Discrimination

IEEE Transactions on Pattern Analysis and Machine Intelligence
Nearest neighbor pattern classification

IEEE Transactions on Information Theory

Improving nearest neighbor rule with a simple adaptive distance measure

Pattern Recognition Letters
Mean shift-based clustering analysis of multispectral remote sensing imagery

International Journal of Remote Sensing
A method of learning weighted similarity function to improve the performance of nearest neighbor

Information Sciences: an International Journal
A proposed method of local feature-weighting to improve predictions of basic nearest neighbor rule

ASC '07 Proceedings of The Eleventh IASTED International Conference on Artificial Intelligence and Soft Computing
High performance classification of two imagery tasks in the cue-based brain computer interface

PRIB'07 Proceedings of the 2nd IAPR international conference on Pattern recognition in bioinformatics
Incipient fault diagnosis of rolling element bearing based on wavelet packet transform and energy operator

WSEAS TRANSACTIONS on SYSTEMS
Efficient model selection for large-scale nearest-neighbor data mining

BNCOD'10 Proceedings of the 27th British national conference on Data Security and Security Data
Global geometric similarity scheme for feature selection in fault diagnosis

Expert Systems with Applications: An International Journal
A niching genetic programming-based multi-objective algorithm for hybrid data classification

Neurocomputing

Quantified Score

Hi-index	0.01

Visualization

Abstract

The k-nearest-neighbor rule is one of the most attractive pattern classification algorithms. In practice, the choice of k is determined by the cross-validation method. In this work, we propose a new method for neighborhood size selection that is based on the concept of statistical confidence. We define the confidence associated with a decision that is made by the majority rule from a finite number of observations and use it as a criterion to determine the number of nearest neighbors needed. The new algorithm is tested on several real-world datasets and yields results comparable to the k-nearest-neighbor rule. However, in contrast to the k-nearest-neighbor rule that uses a fixed number of nearest neighbors throughout the feature space, our method locally adjusts the number of nearest neighbors until a satisfactory level of confidence is reached. In addition, the statistical confidence provides a natural way to balance the trade-off between the reject rate and the error rate by excluding patterns that have low confidence levels. We believe that this property of our method can be of great importance in applications where the confidence with which a decision is made is equally or more important than the overall error rate.