Exact bootstrap k-nearest neighbor learners

Authors:
Brian M. Steele
Affiliations:
Dept. of Mathematical Sciences, University of Montana, Missoula, USA 59812
Venue:
Machine Learning
Year:
2009

Citing 10
Cited 3

Original Contribution: Stacked generalization

Neural Networks
On linear discriminant analysis with adaptive ridge classification rules

Journal of Multivariate Analysis
Bagging predictors

Machine Learning
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Pattern Recognition and Neural Networks

Pattern Recognition and Neural Networks
Random Forests

Machine Learning
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Machine Learning
Bagging Equalizes Influence

Machine Learning
Combining bagging and random subspaces to create better ensembles

IDA'07 Proceedings of the 7th international conference on Intelligent data analysis
An empirical evaluation of bagging and boosting

AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence

On the Rate of Convergence of the Bagged Nearest Neighbor Estimate

The Journal of Machine Learning Research
On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification

Journal of Multivariate Analysis
Differential evolution for optimizing the positioning of prototypes in nearest neighbor classification

Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

Bootstrap aggregation, or bagging, is a method of reducing the prediction error of a statistical learner. The goal of bagging is to construct a new learner which is the expectation of the original learner with respect to the empirical distribution function. In nearly all cases, the expectation cannot be computed analytically, and bootstrap sampling is used to produce an approximation. The k-nearest neighbor learners are exceptions to this generalization, and exact bagging of many k-nearest neighbor learners is straightforward. This article presents computationally simple and fast formulae for exact bagging of k-nearest neighbor learners and extends exact bagging methods from the conventional bootstrap sampling (sampling n observations with replacement from a set of n observations) to bootstrap sub-sampling schemes (with and without replacement). In addition, a partially exact k-nearest neighbor regression learner is developed. The article also compares the prediction error associated with elementary and exact bagging k-nearest neighbor learners, and several other ensemble methods using a suite of publicly available data sets.