Evolving Neural Networks for Online Reinforcement Learning

  • Authors:
  • Jan Hendrik Metzen;Mark Edgington;Yohannes Kassahun;Frank Kirchner

  • Affiliations:
  • Robotics Lab, German Research Center for Artificial Intelligence (DFKI), Bremen, Germany D-28359;Robotics Group, University of Bremen, Bremen, Germany D-28359;Robotics Group, University of Bremen, Bremen, Germany D-28359;Robotics Lab, German Research Center for Artificial Intelligence (DFKI), Bremen, Germany D-28359 and Robotics Group, University of Bremen, Bremen, Germany D-28359

  • Venue:
  • Proceedings of the 10th international conference on Parallel Problem Solving from Nature: PPSN X
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

For many complex Reinforcement Learning problems with large and continuous state spaces, neuroevolution (the evolution of artificial neural networks) has achieved promising results. This is especially true when there is noise in sensor and/or actuator signals. These results have mainly been obtained in offline learning settings, where the training and evaluation phase of the system are separated. In contrast, in online Reinforcement Learning tasks where the actual performance of the systems during its learning phase matters, the results of neuroevolution are significantly impaired by its purely exploratory nature, meaning that it does not use (i. e. exploit) its knowledge of the performance of single individuals in order to improve its performance during learning. In this paper we describe modifications which significantly improve the online performance of the neuroevolutionary method Evolutionary Acquisition of Neural Topologies (EANT) and discuss the results obtained on two benchmark problems.