Feed-Forward Learning: Fast Reinforcement Learning of Controllers

  • Authors:
  • Marek Musial;Frank Lemke

  • Affiliations:
  • Real-Time Systems and Robotics (PDV), Technische Universität Berlin,;Real-Time Systems and Robotics (PDV), Technische Universität Berlin,

  • Venue:
  • IWINAC '07 Proceedings of the 2nd international work-conference on Nature Inspired Problem-Solving Methods in Knowledge Engineering: Interplay Between Natural and Artificial Computation, Part II
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

Reinforcement Learning (RL) approaches are, very often, rendered useless by the statistics of the required sampling process. This paper shows how very fastRL is essentially made possible by abandoning the state feedback during training episodes. The resulting new method, feed-forward learning(FF learning), employs a return estimator for pairs of a state and a feed-forward policy'sparameter vector. FF learning is particularly suitable for the learning of controllers, e.g. for robotics applications, and yields learning rates unprecedented in the RL context.This paper introduces the method formally and proves a lower bound on its performance. Practical results are provided from applying FF learning to several scenarios based on the collision avoidance behavior of a mobile robot.