A Simple Lexicographic Ranker and Probability Estimator

Authors:
Peter Flach;Edson Takashi Matsubara
Affiliations:
Department of Computer Science, University of Bristol, United Kingdom;Instituto de Ciências e Matemáticas e de Computação, Universidade de São Paulo,
Venue:
ECML '07 Proceedings of the 18th European conference on Machine Learning
Year:
2007

Citing 7
Cited 7

Robust Classification for Imprecise Environments

Machine Learning
Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Learning Decision Trees Using the Area Under the ROC Curve

ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Tree Induction for Probability-Based Ranking

Machine Learning
Properties and benefits of calibrated classifiers

PKDD '04 Proceedings of the 8th European Conference on Principles and Practice of Knowledge Discovery in Databases
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
PAV and the ROC convex hull

Machine Learning

Putting Things in Order: On the Fundamental Role of Ranking in Classification and Probability Estimation

ECML '07 Proceedings of the 18th European conference on Machine Learning
Putting Things in Order: On the Fundamental Role of Ranking in Classification and Probability Estimation

PKDD 2007 Proceedings of the 11th European conference on Principles and Practice of Knowledge Discovery in Databases
Preferences in AI: An overview

Artificial Intelligence
Classifying severely imbalanced data

Canadian AI'11 Proceedings of the 24th Canadian conference on Advances in artificial intelligence
Texture based decision tree classification for Arecanut

Proceedings of the CUBE International Information Technology Conference
A unified view of performance metrics: translating threshold choice into expected classification loss

The Journal of Machine Learning Research
Editorial: Preference learning and ranking

Machine Learning

Quantified Score

Hi-index	0.00

Visualization

Abstract

Given a binary classification task, a ranker sorts a set of instances from highest to lowest expectation that the instance is positive. We propose a lexicographic ranker, LexRank, whose rankings are derived not from scores, but from a simple ranking of attribute values obtained from the training data. When using the odds ratio to rank the attribute values we obtain a restricted version of the naive Bayes ranker. We systematically develop the relationships and differences between classification, ranking, and probability estimation, which leads to a novel connection between the Brier score and ROC curves. Combining LexRankwith isotonic regression, which derives probability estimates from the ROC convex hull, results in the lexicographic probability estimator LexProb. Both LexRankand LexProbare empirically evaluated on a range of data sets, and shown to be highly effective.