SPUDD: stochastic planning using decision diagrams

  • Authors:

  • Jesse Hoey;Robert St-Aubin;Alan Hu;Craig Boutilier

  • Affiliations:

  • Department of Computer Science, University of British Columbia, Vancouver, BC, Canada;Department of Computer Science, University of British Columbia, Vancouver, BC, Canada;Department of Computer Science, University of British Columbia, Vancouver, BC, Canada;Department of Computer Science, University of British Columbia, Vancouver, BC, Canada

  • Venue:
  • UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
  • Year:
  • 1999

Quantified Score

Hi-index 0.01

Visualization

Abstract

Recently, structured methods for solving factored Markov decisions processes (MDPs) with large state spaces have been proposed recently to allow dynamic programming to be applied without the need for complete state enumeration. We propose and examine a new value iteration algorithm for MDPs that uses algebraic decision diagrams (ADDs) to represent value functions and policies, assuming an ADD input representation of the MDP. Dynamic programming is implemented via ADD manipulation. We demonstrate our method on a class of large MDPs (up to 63 million states) and show that significant gains can be had when compared to tree-structured representations (with up to a thirty-fold reduction in the number of nodes required to represent optimal value functions).