The safe bayesian: learning the learning rate via the mixability gap

Authors:
Peter Grünwald
Affiliations:
CWI, Amsterdam and Leiden University, The Netherlands
Venue:
ALT'12 Proceedings of the 23rd international conference on Algorithmic Learning Theory
Year:
2012

Citing 8
Cited 0

Aggregating strategies

COLT '90 Proceedings of the third annual workshop on Computational learning theory
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
PAC-Bayesian Stochastic Model Selection

Machine Learning
Estimation of mixture models

Estimation of mixture models
Pac-bayesian generalisation error bounds for gaussian process classification

The Journal of Machine Learning Research
Suboptimal behavior of Bayes and MDL in classification under misspecification

Machine Learning
Minimum complexity density estimation

IEEE Transactions on Information Theory
Information-theoretic upper and lower bounds for statistical estimation

IEEE Transactions on Information Theory

Quantified Score

Hi-index	0.00

Visualization

Abstract

Standard Bayesian inference can behave suboptimally if the model is wrong. We present a modification of Bayesian inference which continues to achieve good rates with wrong models. Our method adapts the Bayesian learning rate to the data, picking the rate minimizing the cumulative loss of sequential prediction by posterior randomization. Our results can also be used to adapt the learning rate in a PAC-Bayesian context. The results are based on an extension of an inequality due to T. Zhang and others to dependent random variables.