Improving Reinforcement Learning by Using Case Based Heuristics

  • Authors:
  • Reinaldo A. Bianchi;Raquel Ros;Ramon Lopez De Mantaras

  • Affiliations:
  • Centro Universitário da FEI, São Bernardo do Campo, Brazil and Artificial Intelligence Research Institute (IIIA-CSIC), Bellaterra, Spain;Artificial Intelligence Research Institute (IIIA-CSIC), Bellaterra, Spain;Artificial Intelligence Research Institute (IIIA-CSIC), Bellaterra, Spain

  • Venue:
  • ICCBR '09 Proceedings of the 8th International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

This work presents a new approach that allows the use of cases in a case base as heuristics to speed up Reinforcement Learning algorithms, combining Case Based Reasoning (CBR) and Reinforcement Learning (RL) techniques. This approach, called Case Based Heuristically Accelerated Reinforcement Learning (CB-HARL), builds upon an emerging technique, the Heuristic Accelerated Reinforcement Learning (HARL), in which RL methods are accelerated by making use of heuristic information. CB-HARL is a subset of RL that makes use of a heuristic function derived from a case base, in a Case Based Reasoning manner. An algorithm that incorporates CBR techniques into the Heuristically Accelerated Q---Learning is also proposed. Empirical evaluations were conducted in a simulator for the RoboCup Four-Legged Soccer Competition, and results obtained shows that using CB-HARL, the agents learn faster than using either RL or HARL methods.