The value iteration method for countable state Markov decision processes

Authors:
Yossi Aviv;Awi Federgruen
Affiliations:
Olin School of Business, Washington University, St. Louis, MO 63130, USA;Graduate School of Business, Columbia University, New York, NY 10027, USA
Venue:
Operations Research Letters
Year:
1999

Citing 9
Cited 1

An inventory model with limited production capacity and uncertain demands. I. The average-cost criterion

Mathematics of Operations Research
Value iteration in constable state average cost Markov decision processes with unbounded costs

Annals of Operations Research
On the second optimality equation for semi-Markov decision models

Mathematics of Operations Research
Sensitivity analysis for base-stock levels in multiechelon production-inventory systems

Management Science
Markov Decision Processes: Discrete Stochastic Dynamic Programming

Markov Decision Processes: Discrete Stochastic Dynamic Programming
Introduction to Stochastic Dynamic Programming: Probability and Mathematical

Introduction to Stochastic Dynamic Programming: Probability and Mathematical
Stochastic Optimal Control: The Discrete-Time Case

Stochastic Optimal Control: The Discrete-Time Case
Comparing recent assumptions for the existence of average optimal stationary policies

Operations Research Letters
On strong average optimality of markov decision processes with unbounded costs

Operations Research Letters

Average Cost Single-Stage Inventory Models: An Analysis Using a Vanishing Discount Approach

Operations Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper deals with Markov decision processes with a countable state space. We demonstrate that a single, relatively simple condition suffices to guarantee that the value-iteration method converges and that an optimal policy can be computed via this method, once the existence of a solution to the average cost optimality equation has been established via any of the many available sets of existence conditions.