Dynamic reward shaping: training a robot by voice

  • Authors:
  • Ana C. Tenorio-Gonzalez;Eduardo F. Morales;Luis Villaseñor-Pineda

  • Affiliations:
  • National Institute of Astrophysics, Optics and Electronics, Computer Science Department, Tonantzintla, México;National Institute of Astrophysics, Optics and Electronics, Computer Science Department, Tonantzintla, México;National Institute of Astrophysics, Optics and Electronics, Computer Science Department, Tonantzintla, México

  • Venue:
  • IBERAMIA'10 Proceedings of the 12th Ibero-American conference on Advances in artificial intelligence
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Reinforcement Learning is commonly used for learning tasks in robotics, however, traditional algorithms can take very long training times. Reward shaping has been recently used to provide domain knowledge with extra rewards to converge faster. The reward shaping functions are normally defined in advance by the user and are static. This paper introduces a dynamic reward shaping approach, in which these extra rewards are not consistently given, can vary with time and may sometimes be contrary to what is needed for achieving a goal. In the experiments, a user provides verbal feedback while a robot is performing a task which is translated into additional rewards. It is shown that we can still guarantee convergence as long as most of the shaping rewards given per state are consistent with the goals and that even with fairly noisy interaction the system can still produce faster convergence times than traditional reinforcement learning techniques.