Faster algorithms for markov decision processes with low treewidth

  • Authors:
  • Krishnendu Chatterjee;Jakub Łącki

  • Affiliations:
  • IST Austria (Institute of Science and Technology Austria), Austria;Institute of Informatics, University of Warsaw, Poland

  • Venue:
  • CAV'13 Proceedings of the 25th international conference on Computer Aided Verification
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

We consider two core algorithmic problems for probabilistic verification: the maximal end-component decomposition and the almost-sure reachability set computation for Markov decision processes (MDPs). For MDPs with treewidth k, we present two improved static algorithms for both the problems that run in time O(n ·k2.38 ·2k) and O(m ·logn ·k), respectively, where n is the number of states and m is the number of edges, significantly improving the previous known $O(n\cdot k \cdot \sqrt{n\cdot k})$ bound for low treewidth. We also present decremental algorithms for both problems for MDPs with constant treewidth that run in amortized logarithmic time, which is a huge improvement over the previously known algorithms that require amortized linear time.