A response to Webb and Ting's on the application of ROC analysis to predict classification performance under varying class distributions

Authors:
Tom Fawcett;Peter A. Flach
Affiliations:
HP Laboratories, 1501 Page Mill Road, Palo Alto, CA;Department of Computer Science, University of Bristol, Bristol BS8 1UB, UK
Venue:
Machine Learning
Year:
2005

Citing 4
Cited 1

Robust Classification for Imprecise Environments

Machine Learning
Adaptive Fraud Detection

Data Mining and Knowledge Discovery
The Case against Accuracy Estimation for Comparing Induction Algorithms

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
On the Application of ROC Analysis to Predict Classification Performance Under Varying Class Distributions

Machine Learning

Analysis of the contour structural irregularity of skin lesions using wavelet decomposition

Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

In an article in this issue, Webb and Ting criticize ROC analysis for its inability to handle certain changes in class distributions. They imply that the ability of ROC graphs to depict performance in the face of changing class distributions has been overstated. In this editorial response, we describe two general types of domains and argue that Webb and Ting's concerns apply primarily to only one of them. Furthermore, we show that there are interesting real-world domains of the second type, in which ROC analysis may be expected to hold in the face of changing class distributions.