An approximation algorithm for labelled Markov processes: towards realistic approximation

Authors:
Alexandre Bouchard-Cote;Norm Ferns;Prakash Panangaden;Doina Precup
Affiliations:
McGill University Montreal, Quebec, Canada;McGill University Montreal, Quebec, Canada;McGill University Montreal, Quebec, Canada;McGill University Montreal, Quebec, Canada
Venue:
QEST '05 Proceedings of the Second International Conference on the Quantitative Evaluation of Systems
Year:
2005

Citing 0
Cited 5

Approximating and computing behavioural distances in probabilistic transition systems

Theoretical Computer Science
Approximating Markov Processes by Averaging

ICALP '09 Proceedings of the 36th Internatilonal Collogquium on Automata, Languages and Programming: Part II
Continuous time and/or continuous distributions

EPEW'10 Proceedings of the 7th European performance engineering conference on Computer performance engineering
Bisimulation Metrics for Continuous Markov Decision Processes

SIAM Journal on Computing
Approximating Markov Processes by Averaging

Journal of the ACM (JACM)

Quantified Score

Hi-index	0.00

Visualization

Abstract

Approximation techniques for labelled Markov processes on continuous state spaces were developed by Desharnais, Gupta, Jagadeesan and Panangaden. However, it has not been clear whether this scheme could be used in practice since it involves inverting a stochastic kernel.We describe aMonte-Carlobased implementation scheme for this approximation algorithm. This is, to the best of our knowledge, the first implementation of this approximation scheme. The implementation involves some novel ideas about how to estimate infs using sampling and also replacing the explicit description of subsets of the state space by tests for membership. It is hoped that this work will enable more applications of continuous probabilistic LMP theory to emerge.