Effects of Sample Size in Classifier Design
IEEE Transactions on Pattern Analysis and Machine Intelligence
Introduction to statistical pattern recognition (2nd ed.)
Introduction to statistical pattern recognition (2nd ed.)
Comparison of Non-Parametric Methods for Assessing Classifier Performance in Terms of ROC Parameters
AIPR '04 Proceedings of the 33rd Applied Imagery Pattern Recognition Workshop
The Effect of Training Sample Size on Performance of Mass Detection
IWDM '08 Proceedings of the 9th international workshop on Digital Mammography
An experimental decision of samples for RBF neural networks
MUSP'09 Proceedings of the 9th WSEAS international conference on Multimedia systems & signal processing
A pilot sampling method for multi-layer perceptrons
ICCOMP'09 Proceedings of the WSEAES 13th international conference on Computers
The relationship of sample size and accuracy in radial basis function networks
WSEAS Transactions on Computers
Sampling scheme for better RBF network
Proceedings of the 2009 International Conference on Hybrid Information Technology
An empirical improvement of the accuracy of RBF networks
Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human
Empirical determination of sample sizes for multi-layer perceptrons by simple RBF networks
WSEAS Transactions on Computers
An effective sampling scheme for better multi-layer perceptrons
AIKED'10 Proceedings of the 9th WSEAS international conference on Artificial intelligence, knowledge engineering and data bases
Investigating better multi-layer perceptrons for the task of classification
WSEAS Transactions on Computers
An intelligent system for managing the isolation of patients suspected of pulmonary tuberculosis
IDEAL'12 Proceedings of the 13th international conference on Intelligent Data Engineering and Automated Learning
| Hi-index | 0.00 |
In a practical classifier design problem the sample size is limited, and the available finite sample needs to be used both to design a classifier and to predict the classifier's performance for the true population. Since a larger sample is more representative of the population, it is advantageous to design the classifier with all the available cases, and to use a resampling technique for performance prediction. We conducted a Monte Carlo simulation study to compare the ability of different resampling techniques in predicting the performance of a neural network (NN) classifier designed with the available sample. We used the area under the receiver operating characteristic curve as the performance index for the NN classifier. We investigated resampling techniques based on the cross-validation, the leave-one-out method, and three different types of bootstrapping, namely, the ordinary, .632, and .632+ bootstrap. Our results indicated that, under the study conditions, there can be a large difference in the accuracy of the prediction obtained from different resampling methods, especially when the feature space dimensionality is relatively large and the sample size is small. Although this investigation is performed under some specific conditions, it reveals important trends for the problem of classifier performance prediction under the constraint of a limited data set.