Adaptive signal processing
Linear least-squares algorithms for temporal difference learning
Machine Learning - Special issue on reinforcement learning
On the worst-case analysis of temporal-difference learning algorithms
Machine Learning - Special issue on reinforcement learning
Exponentiated gradient versus gradient descent for linear predictors
Information and Computation
COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Numerical Recipes in Pascal: The Art of Scientific Computing
Numerical Recipes in Pascal: The Art of Scientific Computing
IEEE Computational Science & Engineering
Learning to Predict by the Methods of Temporal Differences
Machine Learning
Ridge Regression Learning Algorithm in Dual Variables
ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Least-Squares Temporal Difference Learning
ICML '99 Proceedings of the Sixteenth International Conference on Machine Learning
Worst-case quadratic loss bounds for prediction using linear functions and gradient descent
IEEE Transactions on Neural Networks
| Hi-index | 0.01 |
Foster and Vovk proved relative loss bounds for linear regression where the total loss of the on-line algorithm minus the total loss of the best linear predictor (chosen in hindsight) grows logarithmically with the number of trials. We give similar bounds for temporal-difference learning. Learning takes place in a sequence of trials where the learner tries to predict discounted sums of future reinforcement signals. The quality of the predictions is measured with the square loss and we bound the total loss of the on-line algorithm minus the total loss of the best linear predictor for the whole sequence of trials. Again the difference of the losses is logarithmic in the number of trials. The bounds hold for an arbitrary (worst-case) sequence of examples. We also give a bound on the expected difference for the case when the instances are chosen from an unknown distribution. For linear regression a corresponding lower bound shows that this expected bound cannot be improved substantially.