Optimal threshold policies for operation of a dedicated-platform with imperfect state information – a POMDP framework

Authors:
Arsalan Farrokh;Vikram Krishnamurthy
Affiliations:
University of British Columbia, Vancouver, BC, Canada;University of British Columbia, Vancouver, BC, Canada
Venue:
ECSQARU'05 Proceedings of the 8th European conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
Year:
2005

Citing 3
Cited 0

A survey of algorithmic methods for partially observed Markov decision processes

Annals of Operations Research
Dynamic Programming and Optimal Control

Dynamic Programming and Optimal Control
Introduction to Stochastic Dynamic Programming: Probability and Mathematical

Introduction to Stochastic Dynamic Programming: Probability and Mathematical

Quantified Score

Hi-index	0.00

Visualization

Abstract

We consider the general problem of optimal stochastic control of a dedicated-platform that processes one primary function or task (target-task). The dedicated-platform has two modes of action at each period of time: it can attempt to process the target-task at the given period of time, or suspend the target-task for later completion. We formulate the optimal trade-off between the processing cost and the latency in completion of the target-task as a Partially Observable Markov Decision Process (POMDP). By reformulating this POMDP as a Markovian search problem, we prove that the optimal control policies are threshold in nature. Threshold policies are computationally efficient and inexpensive to implement in real time systems. Numerical results demonstrate the effectiveness of these threshold based operating algorithms as compared to non-optimal heuristic algorithms.