Markov Decision Processes: Discrete Stochastic Dynamic Programming
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Model-Checking Algorithms for Continuous-Time Markov Chains
IEEE Transactions on Software Engineering
Theoretical Computer Science - Tools and algorithms for the construction and analysis of systems (TACAS 2004)
A characterization of meaningful schedulers for continuous-time markov decision processes
FORMATS'06 Proceedings of the 4th international conference on Formal Modeling and Analysis of Timed Systems
Bisimulation and logical preservation for continuous-time markov decision processes
CONCUR'07 Proceedings of the 18th international conference on Concurrency Theory
Stochastic real-time games with qualitative timed automata objectives
CONCUR'10 Proceedings of the 21st international conference on Concurrency theory
The how and why of interactive Markov chains
FMCO'09 Proceedings of the 8th international conference on Formal methods for components and objects
Model checking algorithms for CTMDPs
CAV'11 Proceedings of the 23rd international conference on Computer aided verification
Observing continuous-time MDPs by 1-clock timed automata
RP'11 Proceedings of the 5th international conference on Reachability problems
Model checking interactive markov chains
TACAS'10 Proceedings of the 16th international conference on Tools and Algorithms for the Construction and Analysis of Systems
Model checking: one can do much more than you think!
FSEN'11 Proceedings of the 4th IPM international conference on Fundamentals of Software Engineering
Optimal time-abstract schedulers for CTMDPs and continuous-time Markov games
Theoretical Computer Science
Continuous-time stochastic games with time-bounded reachability
Information and Computation
More or less true: DCTL for continuous-time MDPs
FORMATS'13 Proceedings of the 11th international conference on Formal Modeling and Analysis of Timed Systems
Modelling, reduction and analysis of markov automata
QEST'13 Proceedings of the 10th international conference on Quantitative Evaluation of Systems
Hi-index | 0.00 |
Schedulers in randomly timed games can be classified as to whether they use timing information or not. We consider continuous-time Markov decision processes (CTMDPs) and define a hierarchy of positional (P) and history-dependent (H) schedulers which induce strictly tighter bounds on quantitative properties on CTMDPs. This classification into time abstract (TA), total time (TT) and fully time-dependent (T) schedulers is mainly based on the kind of timing details that the schedulers may exploit. We investigate when the resolution of nondeterminism may be deferred. In particular, we show that TTP and TAP schedulers allow for delaying nondeterminism for all measures, whereas this does neither hold for TP nor for any TAH scheduler. The core of our study is a transformation on CTMDPs which unifies the speed of outgoing transitions per state.