Scheduling networks of queues: heavy traffic analysis of a simple open network
Queueing Systems: Theory and Applications
Discrete flow networks: bottleneck analysis and fluid approximations
Mathematics of Operations Research
Discrete-time controlled Markov processes with average cost criterion: a survey
SIAM Journal on Control and Optimization
Heavy Traffic Convergence of a Controlled, Multiclass Queueing System
SIAM Journal on Control and Optimization
The convergence of value iteration in average cost Markov decision chains
Operations Research Letters
On the value function of a priority queue with an application to a controlled polling model
Queueing Systems: Theory and Applications
Performance Evaluation and Policy Selection in Multiclass Networks
Discrete Event Dynamic Systems
ODE methods for Markov chain stability with applications to MCMC
valuetools '06 Proceedings of the 1st international conference on Performance evaluation methodolgies and tools
Mathematics of Operations Research
| Hi-index | 0.00 |
This paper considers in parallel the scheduling problem for multiclass queueing networks, and optimization of Markov decision processes. It is shown that the value iteration algorithm may perform poorly when the algorithm is not initialized properly. The most typical case where the initial value function is taken to be zero may be a particularly bad choice. In contrast, if the value iteration algorithm is initialized with a stochastic Lyapunov function, then the following hold: (i) a stochastic Lyapunov function exists for each intermediate policy, and hence each policy is regular (a strong stability condition), (ii) intermediate costs converge to the optimal cost, and (iii) any limiting policy is average cost optimal. It is argued that a natural choice for the initial value function is the value function for the associated deterministic control problem based upon a fluid model, or the approximate solution to Poisson’s equation obtained from the LP of Kumar and Meyn. Numerical studies show that either choice may lead to fast convergence to an optimal policy.