Credit Scoring and Its Applications
Credit Scoring and Its Applications
The Case against Accuracy Estimation for Comparing Induction Algorithms
ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
An introduction to ROC analysis
Pattern Recognition Letters - Special issue: ROC analysis in pattern recognition
Mining Supervised Classification Performance Studies: A Meta-Analytic Investigation
Journal of Classification
ROC Curves for Continuous Data
ROC Curves for Continuous Data
Two information-theoretic tools to assess the performance of multi-class classifiers
Pattern Recognition Letters
An application of element oriented analysis based credit scoring
ICDM'10 Proceedings of the 10th industrial conference on Advances in data mining: applications and theoretical aspects
RAMOBoost: ranked minority oversampling in boosting
IEEE Transactions on Neural Networks
Unbiased online active learning in data streams
Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining
A family of measures for best top-n class-selective decision rules
Pattern Recognition
ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) - Special section on ACM multimedia 2010 best paper candidates, and issue on social media
An alternative to ROC and AUC analysis of classifiers
IDA'11 Proceedings of the 10th international conference on Advances in intelligent data analysis X
Using model trees and their ensembles for imbalanced data
CAEPIA'11 Proceedings of the 14th international conference on Advances in artificial intelligence: spanish association for artificial intelligence
Thresholds for error probability measures of business process models
Journal of Systems and Software
Vertex collocation profiles: subgraph counting for link analysis and prediction
Proceedings of the 21st international conference on World Wide Web
The AUK: A simple alternative to the AUC
Engineering Applications of Artificial Intelligence
Performance of classification models from a user perspective
Decision Support Systems
Statistical Analysis and Data Mining
Design principles of massive, robust prediction systems
Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
Expert Systems with Applications: An International Journal
Quality indicators for business process models from a gateway complexity perspective
Information and Software Technology
A New Measure of Classifier Performance for Gene Expression Data
IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
Drift mining in data: A framework for addressing drift in classification
Computational Statistics & Data Analysis
EACL '12 Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics
ACM Transactions on Knowledge Discovery from Data (TKDD) - Special Issue on the Best of SIGKDD 2011
High performance query expansion using adaptive co-training
Information Processing and Management: an International Journal
ROC curve equivalence using the Kolmogorov-Smirnov test
Pattern Recognition Letters
Link Prediction: Fair and Effective Evaluation
ASONAM '12 Proceedings of the 2012 International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2012)
Efficacious end user measures part 1: relative class size and end user problem domains
Advances in Artificial Intelligence - Special issue on Artificial Intelligence Applications in Biomedicine
The Journal of Machine Learning Research
Assessing scorecard performance: A literature review and classification
Expert Systems with Applications: An International Journal
Pattern Recognition
An in-depth study of the potentially confounding effect of class size in fault prediction
ACM Transactions on Software Engineering and Methodology (TOSEM)
Machine Learning
Half-AUC for the evaluation of sensitive or specific classifiers
Pattern Recognition Letters
Computers in Biology and Medicine
A better Beta for the H measure of classification performance
Pattern Recognition Letters
Estimation of a Priori Decision Threshold for Collocations Extraction: An Empirical Study
International Journal of Information Technology and Web Engineering
Wavelet Analysis in Current Cancer Genome Research: A Survey
IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
Hi-index | 0.00 |
The area under the ROC curve (AUC) is a very widely used measure of performance for classification and diagnostic rules. It has the appealing property of being objective, requiring no subjective input from the user. On the other hand, the AUC has disadvantages, some of which are well known. For example, the AUC can give potentially misleading results if ROC curves cross. However, the AUC also has a much more serious deficiency, and one which appears not to have been previously recognised. This is that it is fundamentally incoherent in terms of misclassification costs: the AUC uses different misclassification cost distributions for different classifiers. This means that using the AUC is equivalent to using different metrics to evaluate different classification rules. It is equivalent to saying that, using one classifier, misclassifying a class 1 point is p times as serious as misclassifying a class 0 point, but, using another classifier, misclassifying a class 1 point is P times as serious, where p驴P. This is nonsensical because the relative severities of different kinds of misclassifications of individual points is a property of the problem, not the classifiers which happen to have been chosen. This property is explored in detail, and a simple valid alternative to the AUC is proposed.