Tighter PAC-Bayes bounds through distribution-dependent priors

  • Authors:
  • Guy Lever;François Laviolette;John Shawe-Taylor

  • Affiliations:
  • Department of Computer Science, University College London, Gower Street, London WC1E 6BT, UK;Département dinformatique, Université Laval, Québec, G1K 7P4, Canada;Department of Computer Science, University College London, Gower Street, London WC1E 6BT, UK

  • Venue:
  • Theoretical Computer Science
  • Year:
  • 2013

Quantified Score

Hi-index 5.23

Visualization

Abstract

We further develop the idea that the PAC-Bayes prior can be informed by the data-generating distribution. We use this framework to prove sharp risk bounds for stochastic exponential weights algorithms, and develop insights into controlling function class complexity in this method. In particular we consider controlling capacity with respect to the unknown geometry defined by the data-generating distribution. We also use the method to obtain new bounds for RKHS regularization schemes such as SVMs.