Neural networks and the bias/variance dilemma
Neural Computation
Original Contribution: Stacked generalization
Neural Networks
Neural Computation
Using Iterated Bagging to Debias Regressions
Machine Learning
Machine Learning
Machine Learning
Out of bootstrap estimation of generalization error curves in bagging ensembles
IDEAL'07 Proceedings of the 8th international conference on Intelligent data engineering and automated learning
An evolutionary and attribute-oriented ensemble classifier
ICCSA'06 Proceedings of the 2006 international conference on Computational Science and Its Applications - Volume Part II
Margin distribution based bagging pruning
Neurocomputing
New machine learning algorithm: random forest
ICICA'12 Proceedings of the Third international conference on Information Computing and Applications
| Hi-index | 0.00 |
Bagging (Breiman, 1994a) is a technique that tries toimprove a learning algorithm‘s performance by using bootstrapreplicates of the training set (Efron & Tibshirani, 1993, Efron, 1979). The computational requirements for estimating the resultantgeneralization error on a test set by means of cross-validation areoften prohibitive, for leave-one-out cross-validation one needs totrain the underlying algorithm on the order of mν times, wherem is the size of the training set and ν is the number ofreplicates. This paper presents several techniques for estimatingthe generalization error of a bagged learning algorithm withoutinvoking yet more training of the underlying learning algorithm(beyond that of the bagging itself), as is required bycross-validation-based estimation. These techniques all exploit thebias-variance decomposition (Geman, Bienenstock & Doursat, 1992, Wolpert, 1996). The best of ourestimators also exploits stacking (Wolpert, 1992). In a set ofexperiments reported here, it was found to be more accurate than boththe alternative cross-validation-based estimator of the baggedalgorithm‘s error and the cross-validation-based estimator of theunderlying algorithm‘s error. This improvement was particularlypronounced for small test sets. This suggests a novel justificationfor using bagging—more accurate estimation of the generalizationerror than is possible without bagging.