Computing Poisson probabilities
Communications of the ACM
Modeling and analysis of stochastic systems
Modeling and analysis of stochastic systems
Stochastic dynamic programming and the control of queueing systems
Stochastic dynamic programming and the control of queueing systems
Dynamic power management based on continuous-time Markov decision processes
Proceedings of the 36th annual ACM/IEEE Design Automation Conference
Sequencing Tasks with Exponential Service Times to Minimize the Expected Flow Time or Makespan
Journal of the ACM (JACM)
Model-checking continuous-time Markov chains
ACM Transactions on Computational Logic (TOCL)
Dynamic Programming and Optimal Control, Two Volume Set
Dynamic Programming and Optimal Control, Two Volume Set
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Modelling with Generalized Stochastic Petri Nets
Modelling with Generalized Stochastic Petri Nets
Reduction and Refinement Strategies for Probabilistic Analysis
PAPM-PROBMIV '02 Proceedings of the Second Joint International Workshop on Process Algebra and Probabilistic Methods, Performance Modeling and Verification
Reduction and Refinement Strategies for Probabilistic Analysis
PAPM-PROBMIV '02 Proceedings of the Second Joint International Workshop on Process Algebra and Probabilistic Methods, Performance Modeling and Verification
Probabilistic Symbolic Model Checking with PRISM: A Hybrid Approach
TACAS '02 Proceedings of the 8th International Conference on Tools and Algorithms for the Construction and Analysis of Systems
Model Checking of Probabalistic and Nondeterministic Systems
Proceedings of the 15th Conference on Foundations of Software Technology and Theoretical Computer Science
How to Specify and Verify the Long-Run Average Behavior of Probabilistic Systems
LICS '98 Proceedings of the 13th Annual IEEE Symposium on Logic in Computer Science
Well-defined stochastic Petri nets
MASCOTS '96 Proceedings of the 4th International Workshop on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems
An Efficient Well-Specified Check
PNPM '99 Proceedings of the The 8th International Workshop on Petri Nets and Performance Models
Model-Checking Algorithms for Continuous-Time Markov Chains
IEEE Transactions on Software Engineering
Model checking for a probabilistic branching time logic with fairness
Distributed Computing
Interactive Markov chains: and the quest for quantified quality
Interactive Markov chains: and the quest for quantified quality
On optimal scheduling under uncertainty
TACAS'03 Proceedings of the 9th international conference on Tools and algorithms for the construction and analysis of systems
Abstraction for Stochastic Systems by Erlang's Method of Stages
CONCUR '08 Proceedings of the 19th international conference on Concurrency Theory
Delayed Nondeterminism in Continuous-Time Markov Decision Processes
FOSSACS '09 Proceedings of the 12th International Conference on Foundations of Software Science and Computational Structures: Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2009
CONCUR 2009 Proceedings of the 20th International Conference on Concurrency Theory
Compositional Abstraction for Stochastic Systems
FORMATS '09 Proceedings of the 7th International Conference on Formal Modeling and Analysis of Timed Systems
Probabilistic Model Checking of Biological Systems with Uncertain Kinetic Rates
RP '09 Proceedings of the 3rd International Workshop on Reachability Problems
On a Uniform Framework for the Definition of Stochastic Process Languages
FMICS '09 Proceedings of the 14th International Workshop on Formal Methods for Industrial Critical Systems
Three-valued abstraction for continuous-time Markov chains
CAV'07 Proceedings of the 19th international conference on Computer aided verification
Deciding simulations on probabilistic automata
ATVA'07 Proceedings of the 5th international conference on Automated technology for verification and analysis
A compositional semantics for dynamic fault trees in terms of interactive Markov chains
ATVA'07 Proceedings of the 5th international conference on Automated technology for verification and analysis
Numerical analysis of continuous time Markov decision processes over finite horizons
Computers and Operations Research
Towards a theory of time-bounded verification
ICALP'10 Proceedings of the 37th international colloquium conference on Automata, languages and programming: Part II
Stochastic real-time games with qualitative timed automata objectives
CONCUR'10 Proceedings of the 21st international conference on Concurrency theory
Compositional abstraction of PEPA models for transient analysis
EPEW'10 Proceedings of the 7th European performance engineering conference on Computer performance engineering
Towards analysis of semi-Markov decision processes
AICI'10 Proceedings of the 2010 international conference on Artificial intelligence and computational intelligence: Part I
Time-bounded reachability in tree-structured QBDs by abstraction
Performance Evaluation
The ins and outs of the probabilistic model checker MRMC
Performance Evaluation
Model checking algorithms for CTMDPs
CAV'11 Proceedings of the 23rd international conference on Computer aided verification
Time-bounded verification of CTMCs against real-time specifications
FORMATS'11 Proceedings of the 9th international conference on Formal modeling and analysis of timed systems
Observing continuous-time MDPs by 1-clock timed automata
RP'11 Proceedings of the 5th international conference on Reachability problems
Towards a unified model-based safety assessment
SAFECOMP'06 Proceedings of the 25th international conference on Computer Safety, Reliability, and Security
Probabilistic model checking of biological systems with uncertain kinetic rates
Theoretical Computer Science
Model checking interactive markov chains
TACAS'10 Proceedings of the 16th international conference on Tools and Algorithms for the Construction and Analysis of Systems
Bisimulation and logical preservation for continuous-time markov decision processes
CONCUR'07 Proceedings of the 18th international conference on Concurrency Theory
Playing optimally on timed automata with random delays
FORMATS'12 Proceedings of the 10th international conference on Formal Modeling and Analysis of Timed Systems
Optimal time-abstract schedulers for CTMDPs and continuous-time Markov games
Theoretical Computer Science
Continuous-time stochastic games with time-bounded reachability
Information and Computation
Compositional verification and optimization of interactive markov chains
CONCUR'13 Proceedings of the 24th international conference on Concurrency Theory
More or less true: DCTL for continuous-time MDPs
FORMATS'13 Proceedings of the 11th international conference on Formal Modeling and Analysis of Timed Systems
| Hi-index | 0.00 |
A continuous-time Markov decision process (CTMDP) is a generalization of a continuous-time Markov chain in which both probabilistic and nondeterministic choices co-exist. This paper presents an efficient algorithm to compute the maximum (or minimum) probability to reach a set of goal states within a given time bound in a uniform CTMDP, i.e., a CTMDP in which the delay time distribution per state visit is the same for all states. It furthermore proves that these probabilities coincide for (time-abstract) history-dependent and Markovian schedulers that resolve nondeterminism either deterministically or in a randomized way.