Smooth receiver operating characteristics (smROC) curves

Authors:
William Klement;Peter Flach;Nathalie Japkowicz;Stan Matwin
Affiliations:
School of Electrical Engineering and Computer Science, University of Ottawa, Canada;Computer Science, Bristol University, United Kingdom;School of Electrical Engineering and Computer Science, University of Ottawa, Canada;School of Electrical Engineering and Computer Science, University of Ottawa, Canada and Institute of Computer Science, Polish Academy of Sciences, Warsaw, Poland
Venue:
ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part II
Year:
2011

Citing 10
Cited 0

Tree Induction for Probability-Based Ranking

Machine Learning
Using asymmetric distributions to improve text classifier probability estimates

Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Learning Bayesian network classifiers by maximizing conditional likelihood

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Structural Extension to Logistic Regression: Discriminative Parameter Learning of Belief Net Classifiers

Machine Learning
Learning probabilistic decision trees for AUC

Pattern Recognition Letters - Special issue: ROC analysis in pattern recognition
PAV and the ROC convex hull

Machine Learning
A critical analysis of variants of the AUC

Machine Learning
An Improved Model Selection Heuristic for AUC

ECML '07 Proceedings of the 18th European conference on Machine Learning
AUC: a better measure than accuracy in comparing learning algorithms

AI'03 Proceedings of the 16th Canadian society for computational studies of intelligence conference on Advances in artificial intelligence
Counting positives accurately despite inaccurate classification

ECML'05 Proceedings of the 16th European conference on Machine Learning

Quantified Score

Hi-index	0.00

Visualization

Abstract

Supervised learning algorithms perform common tasks including classification, ranking, scoring, and probability estimation. We investigate how scoring information, often produced by these models, is utilized by an evaluation measure. The ROC curve represents a visualization of the ranking performance of classifiers. However, they ignore the scores which can be quite informative. While this ignored information is less precise than that given by probabilities, it is much more detailed than that conveyed by ranking. This paper presents a novel method to weight the ROC curve by these scores. We call it the Smooth ROC (smROC) curve, and we demonstrate how it can be used to visualize the performance of learning models. We report experimental results to show that the smROC is appropriate for measuring performance similarities and differences between learning models, and is more sensitive to performance characteristics than the standard ROC curve.