Discounted Markov decision processes for small noise intensities

Authors:
Hugo Cruz-Suárez;Rocío Ilhuicatzi-Roldán
Affiliations:
Universidad Autónoma de Puebla, Facultad de Ciencias Físico-Matemáticas, Puebla, México;Universidad Autónoma de Puebla, Facultad de Ciencias Físico-Matemáticas, Puebla, México
Venue:
MATH'09 Proceedings of the 14th WSEAS International Conference on Applied mathematics
Year:
2009

Citing 3
Cited 0

Deterministic Approximation for Stochastic Control Problems

SIAM Journal on Control and Optimization
Adaptive Markov Control Processes

Adaptive Markov Control Processes
Convergence of the Optimal Feedback Policies in a Numerical Method for a Class of Deterministic Optimal Control Problems

SIAM Journal on Control and Optimization

Quantified Score

Hi-index	0.01

Visualization

Abstract

This paper will deal with Markov Decision Processes (MDPs) with an infinite horizon and a total discounted cost. There will be considered a deterministic Markov Decision Process (MDP) and a family of stochastic MDPs indexed by a coefficient Ɛ with values in a certain compact set of real numbers containing zero. For each element of this family the probability law is the transition law of the deterministic MDP perturbed by an additive random noise multiplied by Ɛ. In the paper, there will be provided conditions which guarantee the uniform on compact sets convergence of both the optimal value function and the optimal policy of the stochastic MDP to the optimal value function and the optimal policy of the deterministic one, when Ɛ goes to zero, respectively. The conditions presented in the article will be verified in two examples.