Approximating the multiclass ROC by pairwise analysis

Authors:
Thomas C. W. Landgrebe;Robert P. W. Duin
Affiliations:
Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, P.O. Box 5031 2600 GA Delft, Mekelweg 4 2628CD Delft, The Netherlands;Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, P.O. Box 5031 2600 GA Delft, Mekelweg 4 2628CD Delft, The Netherlands
Venue:
Pattern Recognition Letters
Year:
2007

Citing 9
Cited 3

Robust Classification for Imprecise Environments

Machine Learning
A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems

Machine Learning
Face detection by aggregated Bayesian network classifiers

Pattern Recognition Letters - In memory of Professor E.S. Gelsema
An introduction to ROC analysis

Pattern Recognition Letters - Special issue: ROC analysis in pattern recognition
Multi-class ROC analysis from a multi-objective optimisation perspective

Pattern Recognition Letters - Special issue: ROC analysis in pattern recognition
The mean subjective utility score, a novel metric for cost-sensitive classifier evaluation

Pattern Recognition Letters
Confidence-based classifier design

Pattern Recognition
The use of the area under the ROC curve in the evaluation of machine learning algorithms

Pattern Recognition
Combining accuracy and prior sensitivity for classifier design under prior uncertainty

SSPR'06/SPR'06 Proceedings of the 2006 joint IAPR international conference on Structural, Syntactic, and Statistical Pattern Recognition

The ROC skeleton for multiclass ROC estimation

Pattern Recognition Letters
Comparing multi-objective and threshold-moving ROC curve generation for a prototype-based classifier

Proceedings of the 15th annual conference on Genetic and evolutionary computation
ROC analysis of classifiers in machine learning: A survey

Intelligent Data Analysis

Quantified Score

Hi-index	0.10

Visualization

Abstract

The use of Receiver Operator Characteristic (ROC) analysis for the sake of model selection and threshold optimisation has become a standard practice for the design of two-class pattern recognition systems. Advantages include decision boundary adaptation to imbalanced misallocation costs, the ability to fix some classification errors, and performance evaluation in imprecise, ill-defined conditions where costs, or prior probabilities may vary. Extending this to the multiclass case has recently become a topic of interest. The primary challenge involved is the computational complexity, that increases to the power of the number of classes, rendering many problems intractable. In this paper the multiclass ROC is formalised, and the computational complexities exposed. A pairwise approach is proposed that approximates the multi-dimensional operating characteristic by discounting some interactions, resulting in an algorithm that is tractable, and extensible to large numbers of classes. Two additional multiclass optimisation techniques are also proposed that provide a benchmark for the pairwise algorithm. Experiments compare the various approaches in a variety of practical situations, demonstrating the efficacy of the pairwise approach.