ECML '07 Proceedings of the 18th European conference on Machine Learning
A Simple Lexicographic Ranker and Probability Estimator
ECML '07 Proceedings of the 18th European conference on Machine Learning
PKDD 2007 Proceedings of the 11th European conference on Principles and Practice of Knowledge Discovery in Databases
Calibrated lazy associative classification
SBBD '08 Proceedings of the 23rd Brazilian symposium on Databases
Weakly supervised learning methods for improving the quality of gene name normalization data
ISMB '05 Proceedings of the ACL-ISMB Workshop on Linking Biological Literature, Ontologies and Databases: Mining Biological Semantics
Calibrated lazy associative classification
Information Sciences: an International Journal
Learning speaker, addressee and overlap detection models from multimodal streams
Proceedings of the 14th ACM international conference on Multimodal interaction
A tutorial on human activity recognition using body-worn inertial sensors
ACM Computing Surveys (CSUR)
The Journal of Machine Learning Research
On the effect of calibration in classifier combination
Applied Intelligence
ROC analysis of classifiers in machine learning: A survey
Intelligent Data Analysis
| Hi-index | 0.00 |
A calibrated classifier provides reliable estimates of the true probability that each test sample is a member of the class of interest. This is crucial in decision making tasks. Procedures for calibration have already been studied in weather forecasting, game theory, and more recently in machine learning, with the latter showing empirically that calibration of classifiers helps not only in decision making, but also improves classification accuracy. In this paper we extend the theoretical foundation of these empirical observations. We prove that (1) a well calibrated classifier provides bounds on the Bayes error (2) calibrating a classifier is guaranteed not to decrease classification accuracy, and (3) the procedure of calibration provides the threshold or thresholds on the decision rule that minimize the classification error. We also draw the parallels and differences between methods that use receiver operating characteristic (ROC) curves and calibration based procedures that are aimed at findig a threshold of minimum error. In particular, calibration leads to improved performance when multiple thresholds exist.