Discounted Continuous-Time Markov Decision Processes with Unbounded Rates: The Convex Analytic Approach

Authors:
Alexey Piunovskiy;Yi Zhang
Affiliations:
piunov@liv.ac.uk and zy1985@liv.ac.uk;-
Venue:
SIAM Journal on Control and Optimization
Year:
2011

Citing 5
Cited 1

Stochastic Optimal Control: The Discrete-Time Case

Stochastic Optimal Control: The Discrete-Time Case
Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach

Mathematics of Operations Research
Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces

Mathematics of Operations Research
Brief paper: Towards the optimal control of Markov chains with constraints

Automatica (Journal of IFAC)
Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates

Mathematics of Operations Research

Controlled stochastic jump processes

MMES'10 Proceedings of the 2010 international conference on Mathematical models for engineering science

Quantified Score

Hi-index	0.01

Visualization

Abstract

This paper deals with constrained discounted continuous-time Markov decision processes, also known as controlled jump Markov processes, with Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above and below) cost rates, first, we study the space of occupation measures. Then we reformulate the original problem as a linear program over the space of those measures and undertake the duality analysis. Finally, under some compactness-continuity conditions, we show the existence of a stationary optimal policy out of the class of randomized history-dependent policies.