Multi-class ROC analysis from a multi-objective optimisation perspective

Authors:
Richard M. Everson;Jonathan E. Fieldsend
Affiliations:
Department of Computer Science, School of Engineering, Computer Science and Mathematics, University of Exeter, Harrison Building, Exeter EX4 4QF, UK;Department of Computer Science, School of Engineering, Computer Science and Mathematics, University of Exeter, Harrison Building, Exeter EX4 4QF, UK
Venue:
Pattern Recognition Letters - Special issue: ROC analysis in pattern recognition
Year:
2006

Citing 9
Cited 23

Robust classification systems for imprecise environments

AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems

Machine Learning
Multiobjective Evolutionary Algorithms: Analyzing the State-of-the-Art

Evolutionary Computation
Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy

Evolutionary Computation
The use of the area under the ROC curve in the evaluation of machine learning algorithms

Pattern Recognition
Evolutionary programming made faster

IEEE Transactions on Evolutionary Computation
Using unconstrained elite archives for multiobjective optimization

IEEE Transactions on Evolutionary Computation
Multiobjective optimization of safety related systems: an application to short-term conflict alert

IEEE Transactions on Evolutionary Computation
Nearest neighbor pattern classification

IEEE Transactions on Information Theory

Multiobjective Optimization in Bioinformatics and Computational Biology

IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
Approximating the multiclass ROC by pairwise analysis

Pattern Recognition Letters
ROC analysis in ordinal regression learning

Pattern Recognition Letters
Analysis of Two-Dimensional Non-Rigid Shapes

International Journal of Computer Vision
On the scalability of ordered multi-class ROC analysis

Computational Statistics & Data Analysis
Partial Similarity of Objects, or How to Compare a Centaur to a Horse

International Journal of Computer Vision
Sensitivity and specificity based multiobjective approach for feature selection: Application to cancer diagnosis

Information Processing Letters
A novel measure for evaluating classifiers

Expert Systems with Applications: An International Journal
Paretian similarity for partial comparison of non-rigid objects

SSVM'07 Proceedings of the 1st international conference on Scale space and variational methods in computer vision
The ROC skeleton for multiclass ROC estimation

Pattern Recognition Letters
An adaptive multiobjective approach to evolving ART architectures

IEEE Transactions on Neural Networks
Sensitivity versus accuracy in multiclass problems using memetic Pareto evolutionary neural networks

IEEE Transactions on Neural Networks
Using evolutionary multiobjective techniques for imbalanced classification data

ICANN'10 Proceedings of the 20th international conference on Artificial neural networks: Part I
Learning in the feed-forward random neural network: A critical review

Performance Evaluation
A dynamic over-sampling procedure based on sensitivity for multi-class problems

Pattern Recognition
Visualizing 4D approximation sets of multiobjective optimizers with prosections

Proceedings of the 13th annual conference on Genetic and evolutionary computation
A two-stage evolutionary algorithm based on sensitivity and accuracy for multi-class problems

Information Sciences: an International Journal
Multi-objective learning of Relevance Vector Machine classifiers with multi-resolution kernels

Pattern Recognition
Multi-objective evolutionary optimization for generating ensembles of classifiers in the ROC space

Proceedings of the 14th annual conference on Genetic and evolutionary computation
Ranking data with ordinal labels: optimality and pairwise aggregation

Machine Learning
Comparing multi-objective and threshold-moving ROC curve generation for a prototype-based classifier

Proceedings of the 15th annual conference on Genetic and evolutionary computation
Gene expression rule discovery and multi-objective ROC analysis using a neural-genetic hybrid

International Journal of Data Mining and Bioinformatics
ROC analysis of classifiers in machine learning: A survey

Intelligent Data Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

The receiver operating characteristic (ROC) has become a standard tool for the analysis and comparison of classifiers when the costs of misclassification are unknown. There has been relatively little work, however, examining ROC for more than two classes. Here we discuss and present an extension to the standard two-class ROC for multi-class problems. We define the ROC surface for the Q-class problem in terms of a multi-objective optimisation problem in which the goal is to simultaneously minimise the Q(Q-1) misclassification rates, when the misclassification costs and parameters governing the classifier's behaviour are unknown. We present an evolutionary algorithm to locate the Pareto front-the optimal trade-off surface between misclassifications of different types. The use of the Pareto optimal surface to compare classifiers is discussed and we present a straightforward multi-class analogue of the Gini coefficient. The performance of the evolutionary algorithm is illustrated on a synthetic three class problem, for both k-nearest neighbour and multi-layer perceptron classifiers.