A dynamic allocation method of basis functions in reinforcement learning

  • Authors:
  • Shingo Iida;Kiyotake Kuwayama;Masayoshi Kanoh;Shohei Kato;Hidenori Itoh

  • Affiliations:
  • Dept of Intelligence and Computer Science, Nagoya Institute of Technology, Nagoya, Japan;Dept of Intelligence and Computer Science, Nagoya Institute of Technology, Nagoya, Japan;Dept of System Engineering of Human Body, Chukyo University, Kaizu-cho, Toyota, Japan;Dept of Intelligence and Computer Science, Nagoya Institute of Technology, Nagoya, Japan;Dept of Intelligence and Computer Science, Nagoya Institute of Technology, Nagoya, Japan

  • Venue:
  • AI'04 Proceedings of the 17th Australian joint conference on Advances in Artificial Intelligence
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper, we propose a dynamic allocation method of basis functions, an Allocation/Elimination Gaussian Softmax Basis Function Network (AE-GSBFN), that is used in reinforcement learning AE-GSBFN is a kind of actor-critic method that uses basis functions This method can treat continuous high-dimensional state spaces, because basis functions required only for learning are dynamically allocated, and if an allocated basis function is identified as redundant, the function is eliminated This method overcomes the curse of dimensionality and avoids a fall into local minima through the allocation and elimination processes To confirm the effectiveness of our method, we used a maze task to compare our method with an existing method, which has only an allocation process Moreover, as learning of continuous high-dimensional state spaces, our method was applied to motion control of a humanoid robot We demonstrate that the AE-GSBFN is capable of providing better performance than the existing method.