Synthesis for PCTL in parametric Markov decision processes

  • Authors:
  • Ernst Moritz Hahn;Tingting Han;Lijun Zhang

  • Affiliations:
  • Saarland University, Saarbrücken, Germany;Oxford University Computing Laboratory, United Kingdom;DTU Informatics, Technical University of Denmark, Denmark

  • Venue:
  • NFM'11 Proceedings of the Third international conference on NASA Formal methods
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

In parametric Markov decision processes (PMDPs), transition probabilities are not fixed, but are given as functions over a set of parameters. A PMDP denotes a family of concrete MDPs. This paper studies the synthesis problem for PCTL in PMDPs: Given a specification F in PCTL, we synthesise the parameter valuations under which F is true. First, we divide the possible parameter space into hyper-rectangles. We use existing decision procedures to check whether F holds on each of the Markov processes represented by the hyper-rectangle. As it is normally impossible to cover the whole parameter space by hyper-rectangles, we allow a limited area to remain undecided. We also consider an extension of PCTL with reachability rewards. To demonstrate the applicability of the approach, we apply our technique on a case study, using a preliminary implementation.