Reinforcement Learning for Continuous Stochastic Actions---An Approximation of Probability Density Function by Orthogonal Wave Function Expansion---

  • Authors:
  • Hideki Satoh

  • Affiliations:
  • The author is with the Future University-Hakodate, Hakodate-shi, 041-8655 Japan. E-mail: jamisato@m.ieice.org

  • Venue:
  • IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

A function approximation based on an orthonormal wave function expansion in a complex space is derived. Although a probability density function (PDF) cannot always be expanded in an orthogonal series in a real space because a PDF is a positive real function, the function approximation can approximate an arbitrary PDF with high accuracy. It is applied to an actor-critic method of reinforcement learning to derive an optimal policy expressed by an arbitrary PDF in a continuous-action continuous-state environment. A chaos control problem and a PDF approximation problem are solved using the actor-critic method with the function approximation, and it is shown that the function approximation can approximate a PDF well and that the actor-critic method with the function approximation exhibits high performance.