On the application of ROC analysis to predict classification performance under varying class distributions

Authors:
Geoffrey I. Webb;Kai Ming Ting
Affiliations:
School of Computer Science and Software Engineering, Monash University, P.O. Box 75, Victoria 3800, Australia;Gippsland School of Computing and Information Technology, Monash University, Gippsland Campus Churchill, Victoria 3842, Australia
Venue:
Machine Learning
Year:
2005

Citing 6
Cited 0

Very Simple Classification Rules Perform Well on Most Commonly Used Datasets

Machine Learning
Robust Classification for Imprecise Environments

Machine Learning
Learning Changing Concepts by Exploiting the Structure of Change

Machine Learning
The Case against Accuracy Estimation for Comparing Induction Algorithms

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
The use of the area under the ROC curve in the evaluation of machine learning algorithms

Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

We counsel caution in the application of ROC analysis for prediction of classifier performance under varying class distributions. We argue that it is not reasonable to expect ROC analysis to provide accurate prediction of model performance under varying distributions if the classes contain causally relevant subclasses whose frequencies may vary at different rates or if there are attributes upon which the classes are causally dependent.