Markov decision processes and regular events
Proceedings of the seventeenth international colloquium on Automata, languages and programming
Competitive Markov decision processes
Competitive Markov decision processes
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Stochastic Optimal Control: The Discrete-Time Case
Stochastic Optimal Control: The Discrete-Time Case
How to Specify and Verify the Long-Run Average Behavior of Probabilistic Systems
LICS '98 Proceedings of the 13th Annual IEEE Symposium on Logic in Computer Science
Automata logics, and infinite games: a guide to current research
Automata logics, and infinite games: a guide to current research
LICS '05 Proceedings of the 20th Annual IEEE Symposium on Logic in Computer Science
Games where you can play optimally without any memory
CONCUR 2005 - Concurrency Theory
On the positional determinacy of edge-labeled games
Theoretical Computer Science
Markov decision processes with multiple objectives
STACS'06 Proceedings of the 23rd Annual conference on Theoretical Aspects of Computer Science
Deterministic priority mean-payoff games as limits of discounted games
ICALP'06 Proceedings of the 33rd international conference on Automata, Languages and Programming - Volume Part II
Half-Positional determinacy of infinite games
ICALP'06 Proceedings of the 33rd international conference on Automata, Languages and Programming - Volume Part II
Concurrent games with tail objectives
CSL'06 Proceedings of the 20th international conference on Computer Science Logic
Solving simple stochastic tail games
SODA '10 Proceedings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms
One-counter Markov decision processes
SODA '10 Proceedings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms
| Hi-index | 0.00 |
Markov decision processes (MDPs) are controllable discrete event systems with stochastic transitions. Performances of an MDP are evaluated by a payoff function. The controller of the MDP seeks to optimize those performances, using optimal strategies. There exists various ways of measuring performances, i.e. various classes of payoff functions. For example, average performances can be evaluated by a mean-payoff function, peak performances by a limsup payoff function, and the parity payoff function can be used to encode logical specifications. Surprisingly, all the MDPs equipped with mean, limsup or parity payoff functions share a common non-trivial property: they admit pure stationary optimal strategies. In this paper, we introduce the class of prefix-independent and submixing payoff functions, and we prove that any MDP equipped with such a payoff function admits pure stationary optimal strategies. This result unifies and simplifies several existing proofs. Moreover, it is a key tool for generating new examples of MDPs with pure stationary optimal strategies.