Continuous-time Markov decision processes with nth-bias optimality criteria

Authors:
Junyu Zhang;Xi-Ren Cao
Affiliations:
School of Mathematics and Computational Science, Sun Yat-sen University, Guangzhou, 510275, PR China;Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong
Venue:
Automatica (Journal of IFAC)
Year:
2009

Citing 7
Cited 0

Discrete-time controlled Markov processes with average cost criterion: a survey

SIAM Journal on Control and Optimization
Stochastic dynamic programming and the control of queueing systems

Stochastic dynamic programming and the control of queueing systems
Markov Decision Processes: Discrete Stochastic Dynamic Programming

Markov Decision Processes: Discrete Stochastic Dynamic Programming
From Perturbation Analysis to Markov Decision Processes and Reinforcement Learning

Discrete Event Dynamic Systems
Optimal Control of Ergodic Continuous-Time Markov Chains with Average Sample-Path Rewards

SIAM Journal on Control and Optimization
Bias Optimality for Continuous-Time Controlled Markov Chains

SIAM Journal on Control and Optimization
A unified approach to Markov decision problems and performance sensitivity analysis with discounted and average criteria: multichain cases

Automatica (Journal of IFAC)

Quantified Score

Hi-index	22.14

Visualization

Abstract

In this paper, we study the nth-bias optimality problem for finite continuous-time Markov decision processes (MDPs) with a multichain structure. We first provide nth-bias difference formulas for two policies and present some interesting characterizations of an nth-bias optimal policy by using these difference formulas. Then, we prove the existence of an nth-bias optimal policy by using nth-bias optimal policy iteration algorithms, and show that such an nth-bias optimal policy can be obtained in a finite number of policy iterations.