A Teaching Strategy for Memory-Based Control

  • Authors:
  • John W. Sheppard;Steven L. Salzberg

  • Affiliations:
  • Department of Computer Science, The Johns Hopkins University, Baltimore, Maryland 21218. E-mail: lastname@cs.jhu.edu;Department of Computer Science, The Johns Hopkins University, Baltimore, Maryland 21218. E-mail: lastname@cs.jhu.edu

  • Venue:
  • Artificial Intelligence Review - Special issue on lazy learning
  • Year:
  • 1997

Quantified Score

Hi-index 0.00

Visualization

Abstract

Combining different machine learning algorithms in the same system canproduce benefits above and beyond what either method could achievealone. This paper demonstrates that genetic algorithms can be used inconjunction with lazy learning to solve examples of a difficult class ofdelayed reinforcement learning problems better than either method alone.This class, the class of differential games,includes numerous important control problems that arise in robotics,planning, game playing, and other areas, and solutions for differentialgames suggest solution strategies for the general class of planning and control problems. We conducted a seriesof experiments applying three learning approaches – lazy Q-learning,k-nearest neighbor (k-NN), and a genetic algorithm – to aparticular differential game called a pursuit game. Our experimentsdemonstrate that k-NN had great difficulty solving the problem, while alazy version of Q-learning performed moderately well andthe genetic algorithm performed even better. These resultsmotivated the next step in the experiments, where we hypothesizedk-NN was having difficulty because it did not have good examples – acommon source of difficulty for lazy learning. Therefore, we used thegenetic algorithm as a bootstrapping method for k-NN to createa system to provide these examples. Our experimentsdemonstrate that the resulting joint system learned to solve thepursuit games with a high degree of accuracy – outperforming eithermethod alone – and with relatively small memory requirements.