Two information-theoretic tools to assess the performance of multi-class classifiers

Authors:
Francisco J. Valverde-Albacete;Carmen Peláez-Moreno
Affiliations:
Departamento de Teoría de la Señal y de las Comunicaciones, Universidad Carlos III de Madrid, Avda de la Universidad, 30, 28911 Leganés, Spain;Departamento de Teoría de la Señal y de las Comunicaciones, Universidad Carlos III de Madrid, Avda de la Universidad, 30, 28911 Leganés, Spain
Venue:
Pattern Recognition Letters
Year:
2010

Citing 10
Cited 0

Information-Based Evaluation Criterion for Classifier's Performance

Machine Learning
A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems

Machine Learning
Information Theory, Inference & Learning Algorithms

Information Theory, Inference & Learning Algorithms
An introduction to ROC analysis

Pattern Recognition Letters - Special issue: ROC analysis in pattern recognition
A lot of randomness is hiding in accuracy

Engineering Applications of Artificial Intelligence
A systematic analysis of performance measures for classification tasks

Information Processing and Management: an International Journal
Measuring classifier performance: a coherent alternative to the area under the ROC curve

Machine Learning
The use of the area under the ROC curve in the evaluation of machine learning algorithms

Pattern Recognition
A new outlook on Shannon's information measures

IEEE Transactions on Information Theory
Feature selection in MLPs and SVMs based on maximum output information

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.10

Visualization

Abstract

We develop two tools to analyze the behavior of multiple-class, or multi-class, classifiers by means of entropic measures on their confusion matrix or contingency table. First we obtain a balance equation on the entropies that captures interesting properties of the classifier. Second, by normalizing this balance equation we first obtain a 2-simplex in a three-dimensional entropy space and then the de Finetti entropy diagram or entropy triangle. We also give examples of the assessment of classifiers with these tools.