Average Cost Markov Decision Processes with Weakly Continuous Transition Probabilities

Authors:
Eugene A. Feinberg;Pavlo O. Kasyanov;Nina V. Zadoianchuk
Affiliations:
Department of Applied Mathematics and Statistics, Stony Brook University, Stony Brook, New York 11794;Institute for Applied System Analysis, National Technical University of Ukraine “Kyiv Polytechnic Institute,” Kyiv, Ukraine;Institute for Applied System Analysis, National Technical University of Ukraine “Kyiv Polytechnic Institute,” Kyiv, Ukraine
Venue:
Mathematics of Operations Research
Year:
2012

Citing 7
Cited 0

A counterexample on the optimality equation in Markov decision chains with the average cost criterion

Systems & Control Letters
Average optimality in dynamic programming on Borel spaces: unbounded costs and controls

Systems & Control Letters
Discrete-time controlled Markov processes with average cost criterion: a survey

SIAM Journal on Control and Optimization
Average optimality in dynamic programming with general state space

Mathematics of Operations Research
Stochastic dynamic programming and the control of queueing systems

Stochastic dynamic programming and the control of queueing systems
Optimality of Four-Threshold Policies in Inventory Systems with Customer Returns and Borrowing/Storage Options

Probability in the Engineering and Informational Sciences
Optimality Inequalities for Average Cost Markov Decision Processes and the Stochastic Cash Balance Problem

Mathematics of Operations Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents sufficient conditions for the existence of stationary optimal policies for average cost Markov decision processes with Borel state and action sets and weakly continuous transition probabilities. The one-step cost functions may be unbounded, and the action sets may be noncompact. The main contributions of this paper are: (i) general sufficient conditions for the existence of stationary discount optimal and average cost optimal policies and descriptions of properties of value functions and sets of optimal actions, (ii) a sufficient condition for the average cost optimality of a stationary policy in the form of optimality inequalities, and (iii) approximations of average cost optimal actions by discount optimal actions.