COLT '90 Proceedings of the third annual workshop on Computational learning theory
The nature of statistical learning theory
The nature of statistical learning theory
Journal of the ACM (JACM)
A game of prediction with expert advice
Journal of Computer and System Sciences - Special issue on the eighth annual workshop on computational learning theory, July 5–8, 1995
Combining different procedures for adaptive regression
Journal of Multivariate Analysis
EuroCOLT '99 Proceedings of the 4th European Conference on Computational Learning Theory
How to Better Use Expert Advice
Machine Learning
Sequential prediction of individual sequences under general loss functions
IEEE Transactions on Information Theory
IEEE Transactions on Information Theory
Information-theoretic upper and lower bounds for statistical estimation
IEEE Transactions on Information Theory
Aggregation by exponential weighting and sharp oracle inequalities
COLT'07 Proceedings of the 20th annual conference on Learning theory
Suboptimality of penalized empirical risk minimization in classification
COLT'07 Proceedings of the 20th annual conference on Learning theory
| Hi-index | 0.00 |
We propose a sequential randomized algorithm, which at each step concentrates on functions having both low risk and low variance with respect to the previous step prediction function. It satisfies a simple risk bound, which is sharp to the extent that the standard statistical learning approach, based on supremum of empirical processes, does not lead to algorithms with such a tight guarantee on its efficiency. Our generalization error bounds complement the pioneering work of Cesa-Bianchi et al. [12] in which standard-style statistical results were recovered with tight constants using worst-case analysis. A nice feature of our analysis of the randomized estimator is to put forward the links between the probabilistic and worst-case viewpoint. It also allows to recover recent model selection results due to Juditsky et al. [16] and to improve them in least square regression with heavy noise, i.e. when no exponential moment condition is assumed on the output.