Lazy approximation for solving continuous finite-horizon MDPs

  • Authors:
  • Lihong Li;Michael L. Littman

  • Affiliations:
  • RL3Laboratory, Dept. of Computer Science, Rutgers University, Piscataway, NJ;RL3Laboratory, Dept. of Computer Science, Rutgers University, Piscataway, NJ

  • Venue:
  • AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 3
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Solving Markov decision processes (MDPs) with continuous state spaces is a challenge due to, among other problems. the well-known curse of dimensionality. Nevertheless, numerous real-world applications such as transportation planning and telescope observation scheduling exhibit a critical dependence on continuous states. Current approaches to continuous-state MDPs include discretizing their transition models. In this paper, we propose and study an alternative, discretization-free approach we call lazy approximation. Empirical study shows that lazy approximation performs much better than discretization, and we successfully applied this new technique to a more realistic planetary rover planning problem.