On reoptimizing multi-class classifiers

Authors:
Chris Bourke;Kun Deng;Stephen D. Scott;Robert E. Schapire;N. V. Vinodchandran
Affiliations:
Dept. of Computer Science, University of Nebraska, Lincoln, USA 68588-0115;Dept. of Computer Science, University of Nebraska, Lincoln, USA 68588-0115;Dept. of Computer Science, University of Nebraska, Lincoln, USA 68588-0115;Dept. of Computer Science, Princeton University, Princeton, USA 08540;Dept. of Computer Science, University of Nebraska, Lincoln, USA 68588-0115
Venue:
Machine Learning
Year:
2008

Citing 7
Cited 1

Multiple comparison procedures

Multiple comparison procedures
The Minimum Satisfiability Problem

SIAM Journal on Discrete Mathematics
Robust classification systems for imprecise environments

AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Robust Classification for Imprecise Environments

Machine Learning
A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems

Machine Learning
Convex Optimization

Convex Optimization
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research

The ROC skeleton for multiclass ROC estimation

Pattern Recognition Letters

Quantified Score

Hi-index	0.00

Visualization

Abstract

Significant changes in the instance distribution or associated cost function of a learning problem require one to reoptimize a previously-learned classifier to work under new conditions. We study the problem of reoptimizing a multi-class classifier based on its ROC hypersurface and a matrix describing the costs of each type of prediction error. For a binary classifier, it is straightforward to find an optimal operating point based on its ROC curve and the relative cost of true positive to false positive error. However, the corresponding multi-class problem (finding an optimal operating point based on a ROC hypersurface and cost matrix) is more challenging and until now, it was unknown whether an efficient algorithm existed that found an optimal solution. We answer this question by first proving that the decision version of this problem is $\mathsf{NP}$ -complete. As a complementary positive result, we give an algorithm that finds an optimal solution in polynomial time if the number of classes n is a constant. We also present several heuristics for this problem, including linear, nonlinear, and quadratic programming formulations, genetic algorithms, and a customized algorithm. Empirical results suggest that under both uniform and non-uniform cost models, simple greedy methods outperform more sophisticated methods.