COLT '90 Proceedings of the third annual workshop on Computational learning theory
An introduction to Kolmogorov complexity and its applications
An introduction to Kolmogorov complexity and its applications
STOC '93 Proceedings of the twenty-fifth annual ACM symposium on Theory of computing
The weighted majority algorithm
Information and Computation
Predicting a binary sequence almost as well as the optimal biased coin
COLT '96 Proceedings of the ninth annual conference on Computational learning theory
A game of prediction with expert advice
Journal of Computer and System Sciences - Special issue on the eighth annual workshop on computational learning theory, July 5–8, 1995
Stochastic Complexity in Statistical Inquiry Theory
Stochastic Complexity in Statistical Inquiry Theory
Tight worst-case loss bounds for predicting with expert advice
EuroCOLT '95 Proceedings of the Second European Conference on Computational Learning Theory
Discrete noninformative priors
Discrete noninformative priors
Handbook of Mathematical Functions, With Formulas, Graphs, and Mathematical Tables,
Handbook of Mathematical Functions, With Formulas, Graphs, and Mathematical Tables,
Fisher information and stochastic complexity
IEEE Transactions on Information Theory
Universal portfolios with side information
IEEE Transactions on Information Theory
Minimax redundancy for the class of memoryless sources
IEEE Transactions on Information Theory
A general minimax result for relative entropy
IEEE Transactions on Information Theory
A decision-theoretic extension of stochastic complexity and its applications to learning
IEEE Transactions on Information Theory
The minimum description length principle in coding and modeling
IEEE Transactions on Information Theory
Asymptotic minimax regret for data compression, gambling, and prediction
IEEE Transactions on Information Theory
The context-tree weighting method: basic properties
IEEE Transactions on Information Theory
Superior Guarantees for Sequential Prediction and Lossless Compression via Alphabet Decomposition
The Journal of Machine Learning Research
Regret to the best vs. regret to the average
Machine Learning
Regret to the best vs. regret to the average
COLT'07 Proceedings of the 20th annual conference on Learning theory
| Hi-index | 0.00 |
We apply the exponential weight algorithm, introduced and Littlestone and Warmuth [26] and by Vovk [35] to the problem of predicting a binary sequence almost as well as the best biased coin. We first show that for the case of the logarithmic loss, the derived algorithm is equivalent to the Bayes algorithm with Jeffrey's prior, that was studied by Xie and Barron [38] under probabilistic assumptions. We derive a uniform bound on the regret which holds for any sequence. We also show that if the empirical distribution of the sequence is bounded away from 0 and from 1, then, as the length of the sequence increases to infinity, the difference between this bound and a corresponding bound on the average case regret of the same algorithm (which is asymptotically optimal in that case) is only 1/2. We show that this gap of 1/2 is necessary by calculating the regret of the min-max optimal algorithm for this problem and showing that the asymptotic upper bound is tight. We also study the application of this algorithm to the square loss and show that the algorithm that is derived in this case is different from the Bayes algorithm and is better than it for prediction in the worst-case.