An epsilon-Optimal Grid-Based Algorithm for Partially Observable Markov Decision Processes
ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Predictive state representations: a new theory for modeling dynamical systems
UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Heuristic search value iteration for POMDPs
UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Using predictions for planning and modeling in stochastic environments
Using predictions for planning and modeling in stochastic environments
Improving approximate value iteration using memories and predictive state representations
AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Planning in models that combine memory with predictive representations of state
AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Perseus: randomized point-based value iteration for POMDPs
Journal of Artificial Intelligence Research
Point-based value iteration: an anytime algorithm for POMDPs
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
A planning algorithm for predictive state representations
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Using predictive representations to improve generalization in reinforcement learning
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Model minimization by linear PSR
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Incremental pruning: a simple, fast, exact method for partially observable Markov decision processes
UAI'97 Proceedings of the Thirteenth conference on Uncertainty in artificial intelligence
Closing the learning-planning loop with predictive state representations
Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1
Closing the learning-planning loop with predictive state representations
International Journal of Robotics Research
Goal-Directed online learning of predictive models
EWRL'11 Proceedings of the 9th European conference on Recent Advances in Reinforcement Learning
| Hi-index | 0.00 |
Predictive state representations (PSRs) have been proposed recently as an alternative representation for environments with partial observability. The representation is rooted in actions and observations, so it holds the promise of being easier to learn than Partially Observable Markov Decision Processes (POMDPs). However, comparatively little work has explored planning algorithms using PSRs. Exact methods developed to date are no faster than existing exact planning approaches for POMDPs, and only memory-based PSRs have been shown so far to have an advantage in terms of planning speed. In this paper, we present an algorithm for approximate planning in PSRs, based on an approach similar to point-based value iteration in POMDPs. The point-based approach turns out to be a natural match for the PSR state representation. We present empirical results showing that our approach is either comparable or better than POMDP point-based planning.