Evolution versus temporal difference learning for learning to play Ms. Pac-Man

Authors:
Peter Burrow;Simon M. Lucas
Affiliations:
School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom;School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
Venue:
CIG'09 Proceedings of the 5th international conference on Computational Intelligence and Games
Year:
2009

Citing 12
Cited 0

Adaptation in natural and artificial systems

Adaptation in natural and artificial systems
TD-Gammon, a self-teaching backgammon program, achieves master-level play

Neural Computation
Temporal difference learning and TD-Gammon

Communications of the ACM
An introduction to genetic algorithms

An introduction to genetic algorithms
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Evolving neural networks through augmenting topologies

Evolutionary Computation
Full Automatic ANN Design: A Genetic Approach

IWANN '93 Proceedings of the International Workshop on Artificial Neural Networks: New Trends in Neural Computation
Evolutionary Function Approximation for Reinforcement Learning

The Journal of Machine Learning Research
Ms Pac-Man competition

ACM SIGEVOlution
Learning to play using low-complexity rule-based policies: illustrations through Ms. Pac-Man

Journal of Artificial Intelligence Research
Some studies in machine learning using the game of checkers

IBM Journal of Research and Development
Coevolution versus self-play temporal difference learning for acquiring position evaluation in small-board go

IEEE Transactions on Evolutionary Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper investigates various factors that affect the ability of a system to learn to play Ms. Pac-Man. For this study Ms. Pac-Man provides a game of appropriate complexity, and has the advantage that in recent years there have been many other papers published on systems that learn to play this game. The results indicate that Temporal Difference Learning (TDL) performs most reliably with a tabular function approximator, and that the reward structure chosen can have a dramatic impact on performance. When using a multi-layer perceptron as a function approximator, evolution outperforms TDL by a significant margin. Overall, the best results were obtained by evolving multi-layer perceptrons.