American Mathematical Monthly
Machine Learning
Efficient noise-tolerant learning from statistical queries
Journal of the ACM (JACM)
Robust Classification for Imprecise Environments
Machine Learning
Relative Loss Bounds for Multidimensional Regression Problems
Machine Learning
Divergence function, duality, and convex analysis
Neural Computation
Loss functions, complexities, and the legendre transformation
Theoretical Computer Science - Special issue: Algorithmic learning theory
IEEE Transactions on Pattern Analysis and Machine Intelligence
Neural Computation
Clustering with Bregman Divergences
The Journal of Machine Learning Research
Considering Cost Asymmetry in Learning Classifiers
The Journal of Machine Learning Research
Sparseness vs Estimating Conditional Probabilities: Some Asymptotic Results
The Journal of Machine Learning Research
Eliciting properties of probability distributions
Proceedings of the 9th ACM conference on Electronic commerce
Random classification noise defeats all convex potential boosters
Proceedings of the 25th international conference on Machine learning
Support Vector Machines
Surrogate regret bounds for proper losses
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Bregman Divergences and Surrogates for Learning
IEEE Transactions on Pattern Analysis and Machine Intelligence
The foundations of cost-sensitive learning
IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2
Generalised entropy and asymptotic complexities of languages
COLT'07 Proceedings of the 20th annual conference on Learning theory
Relative loss bounds for single neurons
IEEE Transactions on Neural Networks
Information, Divergence and Risk for Binary Experiments
The Journal of Machine Learning Research
Mixability is bayes risk curvature relative to log loss
The Journal of Machine Learning Research
The Journal of Machine Learning Research
Hi-index | 0.01 |
We study losses for binary classification and class probability estimation and extend the understanding of them from margin losses to general composite losses which are the composition of a proper loss with a link function. We characterise when margin losses can be proper composite losses, explicitly show how to determine a symmetric loss in full from half of one of its partial losses, introduce an intrinsic parametrisation of composite binary losses and give a complete characterisation of the relationship between proper losses and "classification calibrated" losses. We also consider the question of the "best" surrogate binary loss. We introduce a precise notion of "best" and show there exist situations where two convex surrogate losses are incommensurable. We provide a complete explicit characterisation of the convexity of composite binary losses in terms of the link function and the weight function associated with the proper loss which make up the composite loss. This characterisation suggests new ways of "surrogate tuning" as well as providing an explicit characterisation of when Bregman divergences on the unit interval are convex in their second argument. Finally, in an appendix we present some new algorithm-independent results on the relationship between properness, convexity and robustness to misclassification noise for binary losses and show that all convex proper losses are non-robust to misclassification noise.