Introduction to statistical pattern recognition (2nd ed.)
Introduction to statistical pattern recognition (2nd ed.)
Statistical and neural classifiers: an integrated approach to design
Statistical and neural classifiers: an integrated approach to design
A General Model for Finite-Sample Effects in Training and Testing of Competing Classifiers
IEEE Transactions on Pattern Analysis and Machine Intelligence
Comparison of Non-Parametric Methods for Assessing Classifier Performance in Terms of ROC Parameters
AIPR '04 Proceedings of the 33rd Applied Imagery Pattern Recognition Workshop
Estimating the uncertainty in the estimated mean area under the ROC curve of a classifier
Pattern Recognition Letters
Assessment of statistical classification rules: implications for computational intelligence
Assessment of statistical classification rules: implications for computational intelligence
2008 Special Issue: Reader studies for validation of CAD systems
Neural Networks
Derivations of normalized mutual information in binary classifications
FSKD'09 Proceedings of the 6th international conference on Fuzzy systems and knowledge discovery - Volume 1
Uncertainty estimation with a finite dataset in the assessment of classification models
Computational Statistics & Data Analysis
Classifier variability: Accounting for training and testing
Pattern Recognition
Assessing classifiers in terms of the partial area under the ROC curve
Computational Statistics & Data Analysis
| Hi-index | 0.15 |
This paper considers binary classification. We assess a classifier in terms of the Area Under the ROC Curve (AUC). We estimate three important parameters, the conditional AUC (conditional on a particular training set) and the mean and variance of this AUC. We derive, as well, a closed form expression of the variance of the estimator of the AUC. This expression exhibits several components of variance that facilitate an understanding for the sources of uncertainty of that estimate. In addition, we estimate this variance, i.e., the variance of the conditional AUC estimator. Our approach is nonparametric and based on general methods from U--statistics; it addresses the case where the data distribution is neither known nor modeled and where there are only two available data sets, the training and testing sets. Finally, we illustrate some simulation results for these estimators.