Local and global optimization algorithms for generalized learning automata

  • Authors:
  • V. V. Phansalkar;M. A. L. Thathachar

  • Affiliations:
  • -;-

  • Venue:
  • Neural Computation
  • Year:
  • 1995

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper analyzes the long-term behavior of the REINFORCE andrelated algorithms (Williams 1986, 1988, 1992) for generalizedlearning automata (Narendra and Thathachar 1989) for theassociative reinforcement learning problem (Barto and Anandan1985). The learning system considered here is a feedforwardconnectionist network of generalized learning automata units. Weshow that REINFORCE is a gradient ascent algorithm but can exhibitunbounded behavior. A modified version of this algorithm, based onconstrained optimization techniques, is suggested to overcome thisdisadvantage. The modified algorithm is shown to exhibit localoptimization properties. A global version of the algorithm, basedon constant temperature heat bath techniques, is also described andshown to converge to the global maximum. All algorithms areanalyzed using weak convergence techniques.