The value of observation for monitoring dynamic systems

Authors:
Eyal Even-Dar;Sham M. Kakade;Yishay Mansour
Affiliations:
Computer and Information Science, University of Pennsylvania, Philadelphia, PA;Toyota Technological Institute, Chicago, IL;School of Computer Science, Tel Aviv University, Tel Aviv, Israel
Venue:
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Year:
2007

Citing 5
Cited 2

A model for reasoning about persistence and causation

Computational Intelligence
A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes

Machine Learning
Approximate planning for factored POMDPs using belief state simplification

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Factored particles for scalable monitoring

UAI'02 Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence
Tractable inference for complex stochastic processes

UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence

Coordination and multi-tasking using EMT

AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 1
A spectral algorithm for learning Hidden Markov Models

Journal of Computer and System Sciences

Quantified Score

Hi-index	0.00

Visualization

Abstract

We consider the fundamental problem of monitoring (i.e. tracking) the belief state in a dynamic system, when the model is only approximately correct and when the initial belief state might be unknown. In this general setting where the model is (perhaps only slightly) mis-specified, monitoring (and consequently planning) may be impossible as errors might accumulate over time. We provide a new characterization, the value of observation, which allows us to bound the error accumulation. The value of observation is a parameter that governs how much information the observation provides. For instance, in Partially Observable MDPs when it is 1 the POMDP is an MDP while for an unobservable Markov Decision Process the parameter is 0. Thus, the new parameter characterizes a spectrum from MDPs to unobservable MDPs depending on the amount of information conveyed in the observations.