Hierarchical controller learning in a first-person shooter

  • Authors:
  • Niels van Hoorn;Julian Togelius;Jürgen Schmidhuber

  • Affiliations:
  • IDSIA, Manno-Lugano, Switzerland;IT University of Copenhagen, Copenhagen S, Denmark;IDSIA, Manno-Lugano, Switzerland

  • Venue:
  • CIG'09 Proceedings of the 5th international conference on Computational Intelligence and Games
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

We describe the architecture of a hierarchical learning-based controller for bots in the First-Person Shooter (FPS) game Unreal Tournament 2004. The controller is inspired by the subsumption architecture commonly used in behaviour-based robotics. A behaviour selector decides which of three sub-controllers gets to control the bot at each time step. Each controller is implemented as a recurrent neural network, and trained with artificial evolution to perform respectively combat, exploration and path following. The behaviour selector is trained with a multiobjective evolutionary algorithm to achieve an effective balancing of the lower-level behaviours. We argue that FPS games provide good environments for studying the learning of complex behaviours, and that the methods proposed here can help developing interesting opponents for games.