Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity

Authors:
A. Dalalyan;A. B. Tsybakov
Affiliations:
LPMA, University of Paris 6, Paris cedex 05, France 75252;LPMA, University of Paris 6, Paris cedex 05, France 75252 and Laboratoire de Statistique, CREST, Malakoff cedex, France 92240
Venue:
Machine Learning
Year:
2008

Citing 13
Cited 9

Aggregating strategies

COLT '90 Proceedings of the third annual workshop on Computational learning theory
The weighted majority algorithm

Information and Computation
How to use expert advice

Journal of the ACM (JACM)
Combining different procedures for adaptive regression

Journal of Multivariate Analysis
Recursive Aggregation of Estimators by the Mirror Descent Algorithm with Averaging

Problems of Information Transmission
Prediction, Learning, and Games

Prediction, Learning, and Games
Aggregation by exponential weighting and sharp oracle inequalities

COLT'07 Proceedings of the 20th annual conference on Learning theory
Aggregation and sparsity via ℓ1 penalized least squares

COLT'06 Proceedings of the 19th annual conference on Learning Theory
A randomized online learning algorithm for better variance control

COLT'06 Proceedings of the 19th annual conference on Learning Theory
On the generalization ability of on-line learning algorithms

IEEE Transactions on Information Theory
Stable recovery of sparse overcomplete representations in the presence of noise

IEEE Transactions on Information Theory
Information-theoretic upper and lower bounds for statistical estimation

IEEE Transactions on Information Theory
Information Theory and Mixing Least-Squares Regressions

IEEE Transactions on Information Theory

Closed-form MMSE estimation for signal denoising under sparse representation modeling over a unitary dictionary

IEEE Transactions on Signal Processing
A PAC-bayes bound for tailored density estimation

ALT'10 Proceedings of the 21st international conference on Algorithmic learning theory
Competing against the best nearest neighbor filter in regression

ALT'11 Proceedings of the 22nd international conference on Algorithmic learning theory
Anisotropic non-local means with spatially adaptive patch shapes

SSVM'11 Proceedings of the Third international conference on Scale Space and Variational Methods in Computer Vision
Non-local Methods with Shape-Adaptive Patches (NLM-SAP)

Journal of Mathematical Imaging and Vision
Non-convex penalized estimation in high-dimensional models with single-index structure

Journal of Multivariate Analysis
Sparse regression learning by aggregation and Langevin Monte-Carlo

Journal of Computer and System Sciences
Sparse single-index model

The Journal of Machine Learning Research
Sparsity regret bounds for individual sequences in online linear regression

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

We study the problem of aggregation under the squared loss in the model of regression with deterministic design. We obtain sharp PAC-Bayesian risk bounds for aggregates defined via exponential weights, under general assumptions on the distribution of errors and on the functions to aggregate. We then apply these results to derive sparsity oracle inequalities.