Quantitative timed analysis of interactive markov chains
NFM'12 Proceedings of the 4th international conference on NASA Formal Methods
Playing optimally on timed automata with random delays
FORMATS'12 Proceedings of the 10th international conference on Formal Modeling and Analysis of Timed Systems
Optimal time-abstract schedulers for CTMDPs and continuous-time Markov games
Theoretical Computer Science
Hi-index | 0.00 |
We establish the existence of optimal scheduling strategies for time-bounded reachability in continuous-time Markov decision processes, and of co-optimal strategies for continuous-time Markov games. Furthermore, we show that optimal control does not only exist, but has a surprisingly simple structure: the optimal schedulers from our proofs are deterministic and timed positional, and the bounded time can be divided into a finite number of intervals, in which the optimal strategies are positional. That is, we demonstrate the existence of finite optimal control. Finally, we show that these pleasant properties of Markov decision processes extend to the more general class of continuous-time Markov games, and that both early and late schedulers show this behaviour.