On the markovian randomized strategy of controller for markov decision processes

Authors:
Taolue Chen;Tingting Han;Jian Lu
Affiliations:
Department of Software Engineering, CWI, Amsterdam, GB, The Netherlands;State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, Jiangsu, P.R. China;State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, Jiangsu, P.R. China
Venue:
FSKD'06 Proceedings of the Third international conference on Fuzzy Systems and Knowledge Discovery
Year:
2006

Citing 7
Cited 1

Complexity of deciding Tarski algebra

Journal of Symbolic Computation
The complexity of probabilistic verification

Journal of the ACM (JACM)
On the combinatorial and algebraic complexity of quantifier elimination

Journal of the ACM (JACM)
Competitive Markov decision processes

Competitive Markov decision processes
Module checking

Information and Computation - Special issue on FLOC '96
Model Checking of Probabalistic and Nondeterministic Systems

Proceedings of the 15th Conference on Foundations of Software Technology and Theoretical Computer Science
Infinite Games and Verification (Extended Abstract of a Tutorial)

CAV '02 Proceedings of the 14th International Conference on Computer Aided Verification

Extending a Synthesis-Centric Model-Based Systems Engineering Framework with Stochastic Model Checking

Electronic Notes in Theoretical Computer Science (ENTCS)

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper focuses on the so called controller synthesis problem, which addresses the question of how to limit the internal behavior of a given system implementation to meet its specification, regardless of the behavior enforced by the environment. We consider this problem in the probabilistic setting, where the underlying model has both probabilism and nondeterminism and the nondeterministic choices in some states are assumed to be controllable while the others are under the control of an unpredictable environment. As for the specification, it is defined by probabilistic computation tree logic. We show that under the restriction that the controller exploits only Markovian randomized strategy, the existence of such a controller is decidable, which is done by a reduction to the decidability of first-order theory for reals. This also gives rise to an algorithm which can synthesize the controller if it does exist.