Time aggregated Markov decision processes via standard dynamic programming

Authors:
Edilson F. Arruda;Marcelo D. Fragoso
Affiliations:
School of Engineering, Pontifical Catholic University of Rio Grande do Sul, Brazil;Center for Systems and Control-CSC, National Laboratory for Scientific Computation-LNCC. Av. Getúlio Vargas, 333. Petrópolis, RJ 25651-075, Brazil
Venue:
Operations Research Letters
Year:
2011

Citing 3
Cited 0

Markov Decision Processes: Discrete Stochastic Dynamic Programming

Markov Decision Processes: Discrete Stochastic Dynamic Programming
Exact finite approximations of average-cost countable Markov decision processes

Automatica (Journal of IFAC)
A time aggregation approach to Markov decision processes

Automatica (Journal of IFAC)

Quantified Score

Hi-index	0.00

Visualization

Abstract

This note addresses the time aggregation approach to ergodic finite state Markov decision processes with uncontrollable states. We propose the use of the time aggregation approach as an intermediate step toward constructing a transformed MDP whose state space is comprised solely of the controllable states. The proposed approach simplifies the iterative search for the optimal solution by eliminating the need to define an equivalent parametric function, and results in a problem that can be solved by simpler, standard MDP algorithms.