State space segmentation for acquisition of agent behavior

  • Authors:
  • Hiroaki Ueda;Takeshi Naraki;Yo Nasu;Kazuki Mikami;Kenichi Takahashi;Tetsuhiro Miyahara

  • Affiliations:
  • (Correspd. Tel: +81-82-830-1758, E-mail: ueda@its.hiroshima-cu.ac.jp) Graduate School of Information Sciences, Hiroshima City University, Hiroshima 731-3194, Japan;Mazda Motor Corp.;Graduate School of Information Sciences, Hiroshima City University, Hiroshima 731-3194, Japan;NEC System Technologies, Ltd.;Graduate School of Information Sciences, Hiroshima City University, Hiroshima 731-3194, Japan;Graduate School of Information Sciences, Hiroshima City University, Hiroshima 731-3194, Japan

  • Venue:
  • Web Intelligence and Agent Systems
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

We propose a new method SSED (State Segmentation based on Euclidean Distance) to categorize continuous numeric percepts for Q-learning, where percept vectors are classified into categories and Q-learning uses categories as states to acquire rules for agent behavior. In SSED, categories are represented by hyper-spheres. A percept vector is classified into a category that covers the vector and is the nearest to it. For efficient reinforcement learning, category merging is provided with SSED, where the number of parameters to control category merging in SSED is fewer than that in fuzzy ART with category merging. In addition, match tracking is incorporated into SSED in order to specialize a category. SSED is combined with Q-learning and it is compared with some state segmentation methods. Experimental results show that Q-learning with SSED learns good rules for agent behavior more efficiently than other methods.