On strategy improvement algorithms for simple stochastic games

Authors:
Rahul Tripathi;Elena Valkanova;V. S. Anil Kumar
Affiliations:
Department of Computer Science and Engineering, University of South Florida, Tampa;Department of Computer Science and Engineering, University of South Florida, Tampa;Virginia Bioinformatics Institute and Dept. of Computer Science, Virginia Tech., Blacksburg
Venue:
CIAC'10 Proceedings of the 7th international conference on Algorithms and Complexity
Year:
2010

Citing 8
Cited 1

The complexity of stochastic games

Information and Computation
A subexponential randomized algorithm for the simple stochastic game problem

Information and Computation
Finite State Markovian Decision Processes

Finite State Markovian Decision Processes
A Discrete Strategy Improvement Algorithm for Solving Parity Games

CAV '00 Proceedings of the 12th International Conference on Computer Aided Verification
Simple Stochastic Games, Parity Games, Mean Payoff Games and Discounted Payoff Games Are All LP-Type Problems

Algorithmica
An Exponential Lower Bound for the Parity Game Strategy Improvement Algorithm as We Know it

LICS '09 Proceedings of the 2009 24th Annual IEEE Symposium on Logic In Computer Science
New Algorithms for Solving Simple Stochastic Games

Electronic Notes in Theoretical Computer Science (ENTCS)
On the complexity of policy iteration

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence

On strategy improvement algorithms for simple stochastic games

Journal of Discrete Algorithms

Quantified Score

Hi-index	0.00

Visualization

Abstract

The study of simple stochastic games (SSGs) was initiated by Condon for analyzing the computational power of randomized space-bounded alternating Turing machines. The game is played by two players, MAX and MIN, on a directed multigraph, and when the play terminates at a sink s, MAX wins from MIN a payoff p(s)∈[0,1]. Condon showed that the SSG value problem, which given a SSG asks whether the expected payoff won by MAX exceeds 1/2 when both players use their optimal strategies, is in NP ∩ coNP. However, the exact complexity of this problem remains open as it is not known whether the problem is in P or is hard for some natural complexity class. In this paper, we study the computational complexity of a strategy improvement algorithm by Hoffman and Karp for this problem. The Hoffman-Karp algorithm converges to optimal strategies of a given SSG, but no nontrivial bounds were previously known on its running time. We show a bound of O(2n/n) on the convergence time of this algorithm, and a bound of O(20.78 n) on a randomized variant. These are the first non-trivial upper bounds on the convergence time of these strategy improvement algorithms.