Implementing Temporal-Difference Learning with the Scaled Conjugate Gradient Algorithm

  • Authors:
  • Tasos Falas;Andreas Stafylopatis

  • Affiliations:
  • Aff1 Aff2;School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece

  • Venue:
  • Neural Processing Letters
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper investigates the use of the scaled conjugate gradient (SCG) algorithm in temporal-difference (TD) learning for time series prediction. Special emphasis is given on the implementation details, after examining the theoretical background of the algorithm and the learning methodology and how these could be combined. Simple time series (linear, sinusoidal, etc.) as well as more complex ones, coming from real data, are used to examine the behavior of this novel combination of learning algorithm and methodology. Preliminary experimental results indicate that the implementation as presented in this paper indeed works, but the performance (in terms of learning speed and generalization ability) of TD learning using the SCG algorithm is not as good as expected, at least on the representative problems examined. An attempt to rationalize these results is presented.