Predictive state representations: a new theory for modeling dynamical systems
UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Learning predictive state representations in dynamical systems without reset
ICML '05 Proceedings of the 22nd international conference on Machine learning
Observable Operator Models for Discrete Stochastic Time Series
Neural Computation
Planning in models that combine memory with predictive representations of state
AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Perseus: randomized point-based value iteration for POMDPs
Journal of Artificial Intelligence Research
Point-based value iteration: an anytime algorithm for POMDPs
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Combining memory and landmarks with predictive state representations
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Using core beliefs for point-based value iteration
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Model-based online learning of POMDPs
ECML'05 Proceedings of the 16th European conference on Machine Learning
Point-based planning for predictive state representations
Canadian AI'08 Proceedings of the Canadian Society for computational studies of intelligence, 21st conference on Advances in artificial intelligence
Closing the learning-planning loop with predictive state representations
Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
Closing the learning-planning loop with predictive state representations
International Journal of Robotics Research
Goal-Directed online learning of predictive models
EWRL'11 Proceedings of the 9th European conference on Recent Advances in Reinforcement Learning
| Hi-index | 0.00 |
Planning in partially-observable dynamical systems is a challenging problem, and recent developments in point-based techniques, such as Perseus significantly improve performance as compared to exact techniques. In this paper, we show how to apply these techniques to new models for non-Markovian dynamical systems called Predictive State Representatiolls (PSRs) and Memory-PSRs (mPSRs). PSRs and mPSRs are models of non-Markovian decision processes that differ from latent-variable models (e.g. HMMs, POMDPs) by representing state using only observable quantities. Further, mPSRs explicitly represent certain structural properties of the dynamical system that are also relevant to planning. We show how planning techniques can be adapted to leverage this structure to improve performance both in terms of execution time as well as quality of the resulting policy.