Improving temporal difference game agent control using a dynamic exploration rate during control learning

  • Authors:
  • Leo Galway;Darryl Charles;Michaela Black

  • Affiliations:
  • School of Computing and information Engineering at the University of Ulster;School of Computing and information Engineering at the University Ulster;School of Computing and information Engineering at the University of Ulster

  • Venue:
  • CIG'09 Proceedings of the 5th international conference on Computational Intelligence and Games
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper investigates the use of a dynamically generated exploration rate when using a reinforcement learning-based game agent controller within a dynamic digital game environment. Temporal Difference learning has been employed for the real-time generation of reactive game agent behaviors within a variation of classic arcade game Pat-Man. Due to the dynamic nature of the game environment initial experiments made use of static, low value for the exploration rate utilized by action selection during learning. However, further experiments were conducted which dynamically generated a value for the exploration rate prior to learning using a genetic algorithm. Results obtained have shown that an improvement in the overall performance of the game agent controller may be achieved when a dynamic exploration rate is used. In particular, if the use of the genetic algorithm is controlled by a measure of the current performance of the game agent, further gains in the overall performance of the game agent may be achieved.