Sensitivity of constrained Markov decision processes
Annals of Operations Research
Time-sharing policies for controlled Markov chains
Operations Research
Stochastic dynamic programming and the control of queueing systems
Stochastic dynamic programming and the control of queueing systems
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Dynamic Programming: Models and Applications
Dynamic Programming: Models and Applications
Finite State Markovian Decision Processes
Finite State Markovian Decision Processes
On constrained Markov decision processes
Operations Research Letters
Energy-efficient communication protocols
Proceedings of the 39th annual Design Automation Conference
| Hi-index | 0.00 |
A stochastic dynamic program incurs two types of cost: a service cost and a quality of service (delay) cost. The objective is to minimize the expected average service cost, subject to a constraint on the average quality of service cost. When the state space S is finite, we show how to compute an optimal policy for the general constrained problem under weak conditions. The development uses a Lagrange multiplier approach and value iteration. When S is denumerably infinite, we give a method for computation of an optimal policy, using a sequence of approximating finite state problems. The method is illustrated with two computational examples.