COLT '90 Proceedings of the third annual workshop on Computational learning theory
The weighted majority algorithm
Information and Computation
Exponentiated gradient versus gradient descent for linear predictors
Information and Computation
Journal of the ACM (JACM)
Using and combining predictors that specialize
STOC '97 Proceedings of the twenty-ninth annual ACM symposium on Theory of computing
A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Sequential prediction of individual sequences under general loss functions
IEEE Transactions on Information Theory
Predicting Nearly as well as the best Pruning of a Planar Decision Graph
ALT '99 Proceedings of the 10th International Conference on Algorithmic Learning Theory
On-Line Algorithm to Predict Nearly as Well as the Best Pruning of a Decision Tree
Progress in Discovery Science, Final Report of the Japanese Discovery Science Project
Tracking a Small Set of Experts by Mixing Past Posteriors
COLT '01/EuroCOLT '01 Proceedings of the 14th Annual Conference on Computational Learning Theory and and 5th European Conference on Computational Learning Theory
Potential-Based Algorithms in Online Prediction and Game Theory
COLT '01/EuroCOLT '01 Proceedings of the 14th Annual Conference on Computational Learning Theory and and 5th European Conference on Computational Learning Theory
Competing against the best nearest neighbor filter in regression
ALT'11 Proceedings of the 22nd international conference on Algorithmic learning theory
A randomized online learning algorithm for better variance control
COLT'06 Proceedings of the 19th annual conference on Learning Theory
Logarithmic regret algorithms for online convex optimization
COLT'06 Proceedings of the 19th annual conference on Learning Theory
Continuous experts and the binning algorithm
COLT'06 Proceedings of the 19th annual conference on Learning Theory
Sparse regression learning by aggregation and Langevin Monte-Carlo
Journal of Computer and System Sciences
Efficient Market Making via Convex Optimization, and a Connection to Online Learning
ACM Transactions on Economics and Computation - Special Issue on Algorithmic Game Theory
Sparsity regret bounds for individual sequences in online linear regression
The Journal of Machine Learning Research
| Hi-index | 0.00 |
We consider algorithms for combining advice from a set of experts. In each trial, the algorithm receives the predictions of the experts and produces its own prediction. A loss function is applied to measure the discrepancy between the predictions and actual observations. The algorithm keeps a weight for each expert. At each trial the weights are first used to help produce the prediction and then updated according to the observed outcome. Our starting point is Vovk's Aggregating Algorithm, in which the weights have a simple form: the weight of an expert decreases exponentially as a function of the loss incurred by the expert. The prediction of the Aggregating Algorithm is typically a non-linear function of the weights and the experts' predictions. We analyze here a simplified algorithm in which the weights are as in the original Aggregating Algorithm, but the prediction is simply the weighted average of the experts' predictions. We show that for a large class of loss functions, even with the simplified prediction rule the additional loss of the algorithm over the loss of the best expert is at most c ln n, where n is the number of experts and c a constant that depends on the loss function. Thus, the bound is of the same form as the known bounds for the Aggregating Algorithm, although the constants here are not quite as good. We use relative entropy to rewrite the bounds in a stronger form and to motivate the update.