States representations with a hierarchical dependency in reinforcement learning

  • Authors:
  • Sylvain Kamdem;Hideki Iwasaki;Hidehiro Ohki;Naomichi Sueda

  • Affiliations:
  • Oita University, Dannoharu Oita-shi, Japan;Oita University, Dannoharu Oita-shi, Japan;Oita University, Dannoharu Oita-shi, Japan;Oita University, Dannoharu Oita-shi, Japan

  • Venue:
  • ISC '07 Proceedings of the 10th IASTED International Conference on Intelligent Systems and Control
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this research, we focus on the state space construction problem in cases where the state space exhibits some hierarchical dependencies. In the case of the control of a robot's motion, it is believed there exist a strong correlation between the current position and the current speed. For positions close to the goal area, it is necessary to define an area for slowing down whereas a high speed motion is convenient for distant positions. We conduct several experiments to evaluate the effects of combining Self Organizing Map and grid-based representations to autonomously build such hierarchical structures. The results showed that with an extended acceleration range and a more detailed control, the hierarchically built state spaces permitted to achieve a smoother convergence, a better accuracy of learning and a more stable control.