Reinforcement learning based on local state feature learning and policy adjustment

  • Authors:
  • Ya-Ping Lin;Xue-Yong Li

  • Affiliations:
  • College of Computer and Communication, Hunan University, Changsha 410082, China;College of Computer and Communication, Hunan University, Changsha 410082, China

  • Venue:
  • Information Sciences—Informatics and Computer Science: An International Journal - Special issue: Introduction to multimedia and mobile agents
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

The extension of reinforcement learning (RL) to large state space has inevitably encountered the problem of the curse of dimensionality. Improving the learning efficiency of the agent is much more important to the practical application of RL. Consider learning to optimally solve Markov decision problems in a particular domain, if the domain has particular characteristics that are attributable to each state, the agent might be able to take advantage of these features to direct the future learning. This paper firstly defines the local state feature, then a state feature function is used to generate the local state features of a state. Also a weight function is introduced to adjust current policy to the actions worth exploring. Based on the above, an improved SARSA algorithm, Feature-SARSA, is proposed. We validate our new algorithm by experiment on a complex domain, named Sokoban. The results show that the new algorithm has better performance.