Controlled jump Markov processes with local transitions and their fluid approximation

Authors:
Alexey Piunovskiy
Affiliations:
The University of Liverpool, Department of Mathematical Sciences, Peach St. Liverpool, UK
Venue:
WSEAS Transactions on Systems and Control
Year:
2009

Citing 7
Cited 1

Strong approximations for Markovian service networks

Queueing Systems: Theory and Applications
A stability criterion via fluid limits and its application to a polling system

Queueing Systems: Theory and Applications
On the value function of a priority queue with an application to a controlled polling model

Queueing Systems: Theory and Applications
An Explicit Solution for the Value Function of a Priority Queue

Queueing Systems: Theory and Applications
Scheduling Flexible Servers with Convex Delay Costs: Heavy-Traffic Optimality of the Generalized cµ-Rule

Operations Research
Introduction to Probability Models, Ninth Edition

Introduction to Probability Models, Ninth Edition
Revenue Management for a Multiclass Single-Server Queue via a Fluid Model Analysis

Operations Research

Controlled stochastic jump processes

MMES'10 Proceedings of the 2010 international conference on Mathematical models for engineering science

Quantified Score

Hi-index	0.00

Visualization

Abstract

Stochastic jump processes, especially birth-and-death processes, are widely used in the queuing theory, computer networks and information transmission. The state of such process describes the instant length of the queues (numbers of packets at different edges to be transmitted through the net). If the birth and death rates are big, trajectories of such processes are close to the trajectories of deterministic dynamic systems. Therefore, if we consider the related optimal control problems, we expect that the optimal control strategy in the deterministic ('fluid') model will be nearly optimal in the underlying stochastic model. In the current paper, a new technique for calculating the accuracy of this approximation is described. In a nutshell, instead of the study of trajectories, we investigate the corresponding dynamic programming equations. It should be emphasized that we deal also with multiple-dimensional lattices, so that the results are applicable to complex communicating systems of queues. Other areas of application are population dynamics, mathematical epidemiology, and inventory systems.