Stochastic dynamic programming and the control of queueing systems
Stochastic dynamic programming and the control of queueing systems
Dynamic power management based on continuous-time Markov decision processes
Proceedings of the 36th annual ACM/IEEE Design Automation Conference
Sequencing Tasks with Exponential Service Times to Minimize the Expected Flow Time or Makespan
Journal of the ACM (JACM)
Measure and probability for concurrency theorists
Theoretical Computer Science - Special issues on models and paradigms for concurrency
Dynamic Programming and Optimal Control, Two Volume Set
Dynamic Programming and Optimal Control, Two Volume Set
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Verifying Quantitative Properties of Continuous Probabilistic Timed Automata
CONCUR '00 Proceedings of the 11th International Conference on Concurrency Theory
Semi-pullbacks and bisimulation in categories of Markov processes
Mathematical Structures in Computer Science
Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach
Mathematics of Operations Research
On optimal scheduling under uncertainty
TACAS'03 Proceedings of the 9th international conference on Tools and algorithms for the construction and analysis of systems
Stochastic transition systems for continuous state spaces and non-determinism
FOSSACS'05 Proceedings of the 8th international conference on Foundations of Software Science and Computation Structures
Delayed Nondeterminism in Continuous-Time Markov Decision Processes
FOSSACS '09 Proceedings of the 12th International Conference on Foundations of Software Science and Computational Structures: Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2009
Deciding simulations on probabilistic automata
ATVA'07 Proceedings of the 5th international conference on Automated technology for verification and analysis
Towards analysis of semi-Markov decision processes
AICI'10 Proceedings of the 2010 international conference on Artificial intelligence and computational intelligence: Part I
Observing continuous-time MDPs by 1-clock timed automata
RP'11 Proceedings of the 5th international conference on Reachability problems
Bisimulation and logical preservation for continuous-time markov decision processes
CONCUR'07 Proceedings of the 18th international conference on Concurrency Theory
Playing optimally on timed automata with random delays
FORMATS'12 Proceedings of the 10th international conference on Formal Modeling and Analysis of Timed Systems
Optimal time-abstract schedulers for CTMDPs and continuous-time Markov games
Theoretical Computer Science
More or less true: DCTL for continuous-time MDPs
FORMATS'13 Proceedings of the 11th international conference on Formal Modeling and Analysis of Timed Systems
| Hi-index | 0.00 |
Continuous-time Markov decision process are an important variant of labelled transition systems having nondeterminism through labels and stochasticity through exponential fire-time distributions. Nondeterministic choices are resolved using the notion of a scheduler. In this paper we characterize the class of measurable schedulers, which is the most general one, and show how a measurable scheduler induces a unique probability measure on the sigma-algebra of infinite paths. We then give evidence that for particular reachability properties it is sufficient to consider a subset of measurable schedulers. Having analyzed schedulers and their induced probability measures we finally show that each probability measure on the sigma-algebra of infinite paths is indeed induced by a measurable scheduler which proves that this class is complete.