Extensions to the CART algorithm
International Journal of Man-Machine Studies
A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
A study of cross-validation and bootstrap for accuracy estimation and model selection
IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2
Computational Statistics & Data Analysis
Binary trees for dissimilarity data
Computational Statistics & Data Analysis
Computational Statistics & Data Analysis
Fast robust estimation of prediction error based on resampling
Computational Statistics & Data Analysis
An incremental genetic algorithm for classification and sensitivity analysis of its parameters
Expert Systems with Applications: An International Journal
Approximate Bayesian inference in spatial GLMM with skew normal latent variables
Computational Statistics & Data Analysis
Computational Statistics & Data Analysis
Computational Statistics & Data Analysis
A nonparametric classification method based on K-associated graphs
Information Sciences: an International Journal
Uncertainty estimation with a finite dataset in the assessment of classification models
Computational Statistics & Data Analysis
Resampling methods for meta-model validation with recommendations for evolutionary computation
Evolutionary Computation
Wrapper feature selection for small sample size data driven by complete error estimates
Computer Methods and Programs in Biomedicine
Bias-Guided random walk for network-based data classification
ISNN'13 Proceedings of the 10th international conference on Advances in Neural Networks - Volume Part II
Wastewater treatment plant performance prediction with support vector machines
ICDM'13 Proceedings of the 13th international conference on Advances in Data Mining: applications and theoretical aspects
Mean field variational Bayesian inference for support vector machine classification
Computational Statistics & Data Analysis
| Hi-index | 0.03 |
We consider the accuracy estimation of a classifier constructed on a given training sample. The naive resubstitution estimate is known to have a downward bias problem. The traditional approach to tackling this bias problem is cross-validation. The bootstrap is another way to bring down the high variability of cross-validation. But a direct comparison of the two estimators, cross-validation and bootstrap, is not fair because the latter estimator requires much heavier computation. We performed an empirical study to compare the .632+ bootstrap estimator with the repeated 10-fold cross-validation and the repeated one-third holdout estimator. All the estimators were set to require about the same amount of computation. In the simulation study, the repeated 10-fold cross-validation estimator was found to have better performance than the .632+ bootstrap estimator when the classifier is highly adaptive to the training sample. We have also found that the .632+ bootstrap estimator suffers from a bias problem for large samples as well as for small samples.