Partially observable Markov decision processes with imprecise parameters
Artificial Intelligence
Evaluation of a hierarchical reinforcement learning spoken dialogue system
Computer Speech and Language
Online planning algorithms for POMDPs
Journal of Artificial Intelligence Research
AEMS: an anytime online search algorithm for approximate policy refinement in large POMDPs
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Equivalence relations in fully and partially observable Markov decision processes
IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Journal of Intelligent and Robotic Systems
POMDP filter: pruning POMDP value functions with the Kaczmarz iterative method
MICAI'10 Proceedings of the 9th Mexican international conference on Advances in artificial intelligence: Part I
Spatially-aware dialogue control using hierarchical reinforcement learning
ACM Transactions on Speech and Language Processing (TSLP)
Analyzing and escaping local optima in planning as inference for partially observable domains
ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part II
Multi-policy dialogue management
SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
Probabilistic dialogue models with prior domain knowledge
SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Hi-index | 0.00 |
The problem of planning under uncertainty has received significant attention in the scientific community over the past few years. It is now well-recognized that considering uncertainty during planning and decision-making is imperative to the design of robust computer systems. This is particularly crucial in robotics, where the ability to interact effectively with real-world environments is a prerequisite for success. The Partially Observable Markov Decision Process (POMDP) provides a rich framework for planning under uncertainty. The POMDP model can optimize sequences of actions which are robust to sensor noise, missing information, occlusion, as well as imprecise actuators. While the model is sufficiently rich to address most robotic planning problems, exact solutions are generally intractable for all but the smallest problems. This thesis argues that large POMDP problems can be solved by exploiting natural structural constraints. In support of this, we propose two distinct but complementary algorithms which overcome tractability issues in POMDP planning. PBVI is a sample-based approach which approximates a value function solution by planning over a small number of salient information states. PolCA+ is a hierarchical approach which leverages structural properties of a problem to decompose it into a set of smaller, easy-to-solve, problems. These techniques improve the tractability of POMDP planning to the point where POMDP-based robot controllers are a reality. This is demonstrated through the successful deployment of a nursing assistant robot.