Minimizing expected termination time in one-counter markov decision processes

Authors:
Tomáš Brázdil;Antonín Kučera;Petr Novotný;Dominik Wojtczak
Affiliations:
Faculty of Informatics, Masaryk University, Czech Republic;Faculty of Informatics, Masaryk University, Czech Republic;Faculty of Informatics, Masaryk University, Czech Republic;Department of Computer Science, University of Liverpool, UK
Venue:
ICALP'12 Proceedings of the 39th international colloquium conference on Automata, Languages, and Programming - Volume Part II
Year:
2012

Citing 12
Cited 1

Competitive Markov decision processes

Competitive Markov decision processes
The complexity of bisimilarity-checking for one-counter processes

Theoretical Computer Science
DP Lower bounds for equivalence-checking and model-checking of one-counter automata

Information and Computation
Reachability in recursive Markov decision processes

Information and Computation
Recursive Stochastic Games with Positive Rewards

ICALP '08 Proceedings of the 35th international colloquium on Automata, Languages and Programming, Part I
Quasi-Birth-Death Processes, Tree-Like QBDs, Probabilistic 1-Counter Automata, and Pushdown Systems

Performance Evaluation
One-counter Markov decision processes

SODA '10 Proceedings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms
Energy parity games

ICALP'10 Proceedings of the 37th international colloquium conference on Automata, languages and programming: Part II
Approximating the termination value of one-counter MDPS and stochastic games

ICALP'11 Proceedings of the 38th international conference on Automata, languages and programming - Volume Part II
Efficient qualitative analysis of classes of recursive markov decision processes and simple stochastic games

STACS'06 Proceedings of the 23rd Annual conference on Theoretical Aspects of Computer Science
Parity games played on transition graphs of one-counter processes

FOSSACS'06 Proceedings of the 9th European joint conference on Foundations of Software Science and Computation Structures
Recursive markov decision processes and recursive stochastic games

ICALP'05 Proceedings of the 32nd international conference on Automata, Languages and Programming

Playing games with counter automata

RP'12 Proceedings of the 6th international conference on Reachability Problems

Quantified Score

Hi-index	0.00

Visualization

Abstract

We consider the problem of computing the value and an optimal strategy for minimizing the expected termination time in one-counter Markov decision processes. Since the value may be irrational and an optimal strategy may be rather complicated, we concentrate on the problems of approximating the value up to a given error ε0 and computing a finite representation of an ε-optimal strategy. We show that these problems are solvable in exponential time for a given configuration, and we also show that they are computationally hard in the sense that a polynomial-time approximation algorithm cannot exist unless P=NP.