A policy iteration algorithm for Markov decision processes skip-free in one direction

Authors:
J. Lambert;B. Van Houdt;C. Blondia
Affiliations:
University of Antwerp, Middelheimlaan, Antwerp - Belgium;University of Antwerp, Middelheimlaan, Antwerp - Belgium;University of Antwerp, Middelheimlaan, Antwerp - Belgium
Venue:
Proceedings of the 2nd international conference on Performance evaluation methodologies and tools
Year:
2007

Citing 2
Cited 2

Stochastic modelling and analysis: a computational approach

Stochastic modelling and analysis: a computational approach
Dynamic Programming and Optimal Control

Dynamic Programming and Optimal Control

Approximating the termination value of one-counter MDPS and stochastic games

ICALP'11 Proceedings of the 38th international conference on Automata, languages and programming - Volume Part II
Approximating the termination value of one-counter MDPs and stochastic games

Information and Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we present a new algorithm for policy iteration for Markov decision processes (MDP) skip-free in one direction. This algorithm, which is based on matrix analytic methods, is in the same spirit as the algorithm of White (Stochastic Models, 21:785-797, 2005) which was limited to matrices that are skip-free in both directions. Optimization problems that can be solved using Markov decision processes arise in the domain of optical buffers, when trying to improve loss rates of fibre delay line (FDL) buffers. Based on the analysis of such an FDL buffer we present a comparative study between the different techniques available to solve an MDP. The results illustrate that the exploitation of the structure of the transition matrices places us in a position to deal with larger systems, while reducing the computation times.