Computing Poisson probabilities
Communications of the ACM
Spectral transforms for large boolean functions with applications to technology mapping
DAC '93 Proceedings of the 30th international Design Automation Conference
Algebraic decision diagrams and their applications
ICCAD '93 Proceedings of the 1993 IEEE/ACM international conference on Computer-aided design
Load-balancing heuristics and process behavior
SIGMETRICS '86/PERFORMANCE '86 Proceedings of the 1986 ACM SIGMETRICS joint international conference on Computer performance modelling, measurement and evaluation
Stochastic dynamic programming with factored representations
Artificial Intelligence
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Dynamic programming for structured continuous Markov decision problems
UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Solving factored MDPs with continuous and discrete variables
UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Solving generalized semi-Markov decision processes using continuous phase-type distributions
AAAI'04 Proceedings of the 19th national conference on Artifical intelligence
Efficient solution algorithms for factored MDPs
Journal of Artificial Intelligence Research
SPUDD: stochastic planning using decision diagrams
UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
| Hi-index | 0.00 |
We consider a special type of continuous-time Markov decision processes (MDPs) that arise when phase-type distributions are used to model the timing of non-Markovian events and actions. We focus, primarily, on the execution of phase-dependent policies. Phases are introduced into a model to represent relevant execution history, but there is no physical manifestation of phases in the real world. We treat phases as partially observable state features and show how a belief distribution over phase configurations can be derived from observable state features through the use of transient analysis for Markov chains. This results in an efficient method for phase tracking during execution that can be combined with the QMDP value method for POMDPs to make action choices. We also discuss, briefly, how the structure of MDPs with phase transitions can be exploited in structured value iteration with symbolic representation of vectors and matrices.