The thing that we tried didn't work very well: deictic representation in reinforcement learning

  • Authors:
  • Sarah Finney;Natalia H. Gardiol;Leslie Pack Kaelbling;Tim Oates

  • Affiliations:
  • AI Lab, MIT, Cambridge, MA;AI Lab, MIT, Cambridge, MA;AI Lab, MIT, Cambridge, MA;Dept of Computer Science, Univ. of Maryland, BC, Baltimore, MD

  • Venue:
  • UAI'02 Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

Most reinforcement learning methods operate on propositional representations of the world state. Such representations are often intractably large and generalize poorly. Using a deictic representation is believed to be a viable alternative: they promise generalization while allowing the use of existing reinforcement-learning methods. Yet, there are few experiments on learning with deictic representations reported in the literature. In this paper we explore the effectiveness of two forms of deictic representation and a naïve propositional representation in a simple blocks-world domain. We find, empirically, that the deictic representations actually worsen learning performance. We conclude with a discussion of possible causes of these results and strategies for more effective learning in domains with objects.