Relational macros for transfer in reinforcement learning

  • Authors:
  • Lisa Torrey;Jude Shavlik;Trevor Walker;Richard Maclin

  • Affiliations:
  • University of Wisconsin, Madison, WI;University of Wisconsin, Madison, WI;University of Wisconsin, Madison, WI;University of Minnesota, Duluth, MN

  • Venue:
  • ILP'07 Proceedings of the 17th international conference on Inductive logic programming
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

We describe an application of inductive logic programming to transfer learning. Transfer learning is the use of knowledge learned in a source task to improve learning in a related target task. The tasks we work with are in reinforcement-learning domains. Our approach transfers relational macros, which are finite-state machines in which the transition conditions and the node actions are represented by first-order logical clauses. We use inductive logic programming to learn a macro that characterizes successful behavior in the source task, and then use the macro for decision-making in the early learning stages of the target task. Through experiments in the RoboCup simulated soccer domain, we show that Relational Macro Transfer via Demonstration (RMT-D) from a source task can provide a substantial head start in the target task.