PAC-Bayes risk bounds for sample-compressed Gibbs classifiers

Authors:
François Laviolette;Mario Marchand
Affiliations:
Université Laval, Québec, Canada;Université Laval, Québec, Canada
Venue:
ICML '05 Proceedings of the 22nd international conference on Machine learning
Year:
2005

Citing 7
Cited 1

Elements of information theory

Elements of information theory
Sample Compression, Learnability, and the Vapnik-Chervonenkis Dimension

Machine Learning
Some PAC-Bayesian Theorems

Machine Learning - The Eleventh Annual Conference on computational Learning Theory
PAC-Bayesian Stochastic Model Selection

Machine Learning
Generalisation Error Bounds for Sparse Linear Classifiers

COLT '00 Proceedings of the Thirteenth Annual Conference on Computational Learning Theory
Pac-bayesian generalisation error bounds for gaussian process classification

The Journal of Machine Learning Research
The set covering machine

The Journal of Machine Learning Research

Consistency of discrete Bayesian learning

Theoretical Computer Science

Quantified Score

Hi-index	0.00

Visualization

Abstract

We extend the PAC-Bayes theorem to the sample-compression setting where each classifier is represented by two independent sources of information: a compression set which consists of a small subset of the training data, and a message string of the additional information needed to obtain a classifier. The new bound is obtained by using a prior over a data-independent set of objects where each object gives a classifier only when the training data is provided. The new PAC-Bayes theorem states that a Gibbs classifier defined on a posterior over sample-compressed classifiers can have a smaller risk bound than any such (deterministic) sample-compressed classifier.