Temporal difference and policy search methods for reinforcement learning: an empirical comparison

Authors:
Matthew E. Taylor;Shimon Whiteson;Peter Stone
Affiliations:
Department of Computer Sciences, The University of Texas at Austin, Austin, Texas;Department of Computer Sciences, The University of Texas at Austin, Austin, Texas;Department of Computer Sciences, The University of Texas at Austin, Austin, Texas
Venue:
AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2
Year:
2007

Citing 11
Cited 2

Efficient reinforcement learning through symbiotic evolution

Machine Learning - Special issue on reinforcement learning
Reinforcement learning with replacing eligibility traces

Machine Learning - Special issue on reinforcement learning
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Evolving neural networks through augmenting topologies

Evolutionary Computation
Genetic Programming And Multi-agent Layered Learning By Reinforcements

GECCO '02 Proceedings of the Genetic and Evolutionary Computation Conference
Evolving Soccer Keepaway Players Through Task Decomposition

Machine Learning
Co-evolving recurrent neurons learn deep memory POMDPs

GECCO '05 Proceedings of the 7th annual conference on Genetic and evolutionary computation
Comparing evolutionary and temporal difference methods in a reinforcement learning domain

Proceedings of the 8th annual conference on Genetic and evolutionary computation
Evolutionary Function Approximation for Reinforcement Learning

The Journal of Machine Learning Research
Competitive coevolution through evolutionary complexification

Journal of Artificial Intelligence Research
Keepaway soccer: from machine learning testbed to benchmark

RoboCup 2005

A new perspective to the keepaway soccer: the takers

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 3
Learning to dance through interactive evolution

EvoCOMNET'10 Proceedings of the 2010 international conference on Applications of Evolutionary Computation - Volume Part II

Quantified Score

Hi-index	0.00

Visualization

Abstract

Reinforcement learning (RL) methods have become popular in recent years because of their ability to solve complex tasks with minimal feedback. Both genetic algorithms (GAs) and temporal difference (TD) methods have proven effective at solving difficult RL problems, but few rigorous comparisons have been conducted. Thus, no general guidelines describing the methods' relative strengths and weaknesses are available. This paper summarizes a detailed empirical comparison between a GA and a TD method in Keepaway, a standard RL benchmark domain based on robot soccer. The results from this study help isolate the factors critical to the performance of each learning method and yield insights into their general strengths and weaknesses.