Loss functions, complexities, and the legendre transformation

Authors:
Yuri Kalnishkan;Volodya Vovk;Michael V. Vyugin
Affiliations:
Department of Computer Science, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK;Department of Computer Science, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK;Department of Computer Science, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK
Venue:
Theoretical Computer Science - Special issue: Algorithmic learning theory
Year:
2004

Citing 8
Cited 5

Optimal control

Optimal control
Aggregating strategies

COLT '90 Proceedings of the third annual workshop on Computational learning theory
The weighted majority algorithm

Information and Computation
Probability (2nd ed.)

Probability (2nd ed.)
How to use expert advice

Journal of the ACM (JACM)
An introduction to Kolmogorov complexity and its applications (2nd ed.)

An introduction to Kolmogorov complexity and its applications (2nd ed.)
Universal portfolio selection

COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
A game of prediction with expert advice

Journal of Computer and System Sciences - Special issue on the eighth annual workshop on computational learning theory, July 5–8, 1995

Generalised entropy and asymptotic complexities of languages

COLT'07 Proceedings of the 20th annual conference on Learning theory
Supermartingales in prediction with expert advice

Theoretical Computer Science
Composite Binary Losses

The Journal of Machine Learning Research
Mixability is bayes risk curvature relative to log loss

The Journal of Machine Learning Research
Predictive complexity and generalized entropy rate of stationary ergodic processes

ALT'12 Proceedings of the 23rd international conference on Algorithmic Learning Theory

Quantified Score

Hi-index	0.00

Visualization

Abstract

The paper introduces a way of re-constructing a loss function from predictive complexity. We show that a loss function and expectations of the corresponding predictive complexity w.r.t. the Bernoulli distribution are related through the Legendre transformation. It is shown that if two loss functions specify the same complexity then they are equivalent in a strong sense. The expectations are also related to the so-called generalized entropy.