Partially Observed Markov Decision Process Multiarmed Bandits---Structural Results
Mathematics of Operations Research
Optimal Threshold Policies for Multivariate Stopping-Time POMDPs
ECSQARU '09 Proceedings of the 10th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
Optimal threshold policies for multivariate POMDPs in radar resource management
IEEE Transactions on Signal Processing
Cross-layer QoS provisioning for multimedia transmissions in cognitive radio networks
WCNC'09 Proceedings of the 2009 IEEE conference on Wireless Communications & Networking Conference
Distributed sender scheduling for multimedia transmission in wireless mobile peer-to-peer networks
IEEE Transactions on Wireless Communications
Planning for multiple measurement channels in a continuous-state POMDP
Annals of Mathematics and Artificial Intelligence
| Hi-index | 35.69 |
We consider the optimal sensor scheduling problem formulated as a partially observed Markov decision process (POMDP). Due to operational constraints, at each time instant, the scheduler can dynamically select one out of a finite number of sensors and record a noisy measurement of an underlying Markov chain. The aim is to compute the optimal measurement scheduling policy, so as to minimize a cost function comprising of estimation errors and measurement costs. The formulation results in a nonstandard POMDP that is nonlinear in the information state. We give sufficient conditions on the cost function, dynamics of the Markov chain and observation probabilities so that the optimal scheduling policy has a threshold structure with respect to a monotone likelihood ratio (MLR) ordering. As a result, the computational complexity of implementing the optimal scheduling policy is inexpensive. We then present stochastic approximation algorithms for estimating the best linear MLR order threshold policy.