Area under the distance threshold curve as an evaluation measure for probabilistic classifiers

Authors:
Sydney Williams;Michael Harris;Jacob Furst;Daniela Raicu
Affiliations:
Biomedical Engineering, Illinois Institute of Technology, Chicago, Illinois;Computer Science, Sonoma State University, Rohnert Park, California;Computing and Digital Media, DePaul University, Chicago, Illinois;Computing and Digital Media, DePaul University, Chicago, Illinois
Venue:
MLDM'13 Proceedings of the 9th international conference on Machine Learning and Data Mining in Pattern Recognition
Year:
2013

Citing 8
Cited 0

A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems

Machine Learning
A Metric for Distributions with Applications to Image Databases

ICCV '98 Proceedings of the Sixth International Conference on Computer Vision
An introduction to ROC analysis

Pattern Recognition Letters - Special issue: ROC analysis in pattern recognition
Cost curves: An improved method for visualizing classifier performance

Machine Learning
AUC: a statistically consistent and more discriminating measure than accuracy

IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
The use of the area under the ROC curve in the evaluation of machine learning algorithms

Pattern Recognition
Comparing dissimilarity measures for content-based image retrieval

AIRS'08 Proceedings of the 4th Asia information retrieval conference on Information retrieval technology
Building an Ensemble of Probabilistic Classifiers for Lung Nodule Interpretation

ICMLA '11 Proceedings of the 2011 10th International Conference on Machine Learning and Applications and Workshops - Volume 02

Quantified Score

Hi-index	0.00

Visualization

Abstract

Evaluation for probabilistic multiclass systems has predominately been done by converting data into binary classes. While effective in quantifying the classifier performance, binary evaluation causes a loss in ability to distinguish between individual classes. We report that the evaluation of multiclass probabilistic classifiers can be quantified by using the area under the distance threshold curve for multiple distance metrics. We construct our classifiers for evaluation with data from the National Cancer Institute (NCI) Lung Image Database Consortium (LIDC) for the semantic characteristic of malignancy. We conclude that the area under the distance threshold curve can provide a measure of the classifier performance when the classifier has more than two classes and probabilistic predictions.