Risk neutral calibration of classifiers

Authors:
Ron Coleman
Affiliations:
Computer Science Department, Marist College, Poughkeepsie, New York, United States
Venue:
AI'03 Proceedings of the 16th Canadian society for computational studies of intelligence conference on Advances in artificial intelligence
Year:
2003

Citing 5
Cited 0

MetaCost: a general method for making classifiers cost-sensitive

KDD '99 Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining
Machine Learning

Machine Learning
Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Transforming classifier scores into accurate multiclass probability estimates

Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we introduce a new, non-invasive approach to calibrating classifiers in a risk neutral setting based on the worthwhile index, W. We employ a simple Markov chain classifier and show through exhausted tests on data sets from the UCI machine-learning repository how to use the worthwhile index to significantly reduce the incidence of misclassification false positives. We hypothesize our approach to calibration may be applied to other classifier systems with few or no changes since the method involves observing the classifier's external behavior, not its internal mechanisms.