Applications of the self-organising map to reinforcement learning

  • Authors:
  • Andrew James Smith

  • Affiliations:
  • The Division of Informatics, Institute for Adaptive and Neural Computation, University of Edinburgh, 5 Forrest Hill, Edinburgh EH1 2QL, UK

  • Venue:
  • Neural Networks - New developments in self-organizing maps
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

This article is concerned with the representation and generalisation of continuous action spaces in reinforcement learning (RL) problems. A model is proposed based on the self-organising map (SOM) of Kohonen [Self Organisation and Associative Memory, 1987] which allows either the one-to-one, many-to-one or one-to-many structure of the desired state-action mapping to be captured. Although presented here for tasks involving immediate reward, the approach is easily extended to delayed reward. We conclude that the SOM is a useful tool for providing real-time, on-line generalisation in RL problems in which the latent dimensionalities of the state and action spaces are small. Scalability issues are also discussed.