Discrete-time, Discrete-valued Observable Operator Models: a Tutorial
Discrete-time, Discrete-valued Observable Operator Models: a Tutorial
Learning and discovery of predictive state representations in dynamical systems with reset
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Predictive state representations: a new theory for modeling dynamical systems
UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Learning topological maps with weak local odometric information
IJCAI'97 Proceedings of the Fifteenth international joint conference on Artifical intelligence - Volume 2
Blind construction of optimal nonlinear recursive predictors for discrete sequences
UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Predictive state representations: a new theory for modeling dynamical systems
UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Relating reinforcement learning performance to classification performance
ICML '05 Proceedings of the 22nd international conference on Machine learning
TD(λ) networks: temporal-difference networks with eligibility traces
ICML '05 Proceedings of the 22nd international conference on Machine learning
Learning predictive representations from a history
ICML '05 Proceedings of the 22nd international conference on Machine learning
Learning predictive state representations in dynamical systems without reset
ICML '05 Proceedings of the 22nd international conference on Machine learning
Learning predictive state representations using non-blind policies
ICML '06 Proceedings of the 23rd international conference on Machine learning
Dealing with non-stationary environments using context detection
ICML '06 Proceedings of the 23rd international conference on Machine learning
Predictive linear-Gaussian models of controlled stochastic dynamical systems
ICML '06 Proceedings of the 23rd international conference on Machine learning
Predictive state representations with options
ICML '06 Proceedings of the 23rd international conference on Machine learning
Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems
Approximate predictive state representations
Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
Dynamics based control with PSRs
Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Representing systems with hidden state
AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Improving approximate value iteration using memories and predictive state representations
AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Learning the Difference between Partially Observable Dynamical Systems
ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II
Planning in models that combine memory with predictive representations of state
AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Improving anytime point-based value iteration using principled point selections
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Grounding abstractions in predictive state representations
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Combining memory and landmarks with predictive state representations
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Temporal-difference networks with history
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Model minimization by linear PSR
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Point-based planning for predictive state representations
Canadian AI'08 Proceedings of the Canadian Society for computational studies of intelligence, 21st conference on Advances in artificial intelligence
Temporal-difference networks for dynamical systems with continuous observations and actions
UAI '09 Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
Closing the learning-planning loop with predictive state representations
International Journal of Robotics Research
A Monte-Carlo AIXI approximation
Journal of Artificial Intelligence Research
Using learned PSR model for planning under uncertainty
AI'10 Proceedings of the 23rd Canadian conference on Advances in Artificial Intelligence
Learning to make predictions in partially observable environments without a generative model
Journal of Artificial Intelligence Research
Goal-Directed online learning of predictive models
EWRL'11 Proceedings of the 9th European conference on Recent Advances in Reinforcement Learning
Feature reinforcement learning in practice
EWRL'11 Proceedings of the 9th European conference on Recent Advances in Reinforcement Learning
The duality of state and observation in probabilistic transition systems
TbiLLC'11 Proceedings of the 9th international conference on Logic, Language, and Computation
Multi-timescale nexting in a reinforcement learning robot
Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
| Hi-index | 0.00 |
Modeling dynamical systems, both for control purposes and to make predictions about their behavior, is ubiquitous in science and engineering. Predictive state representations (PSRs) are a recently introduced class of models for discrete-time dynamical systems. The key idea behind PSRs and the closely related OOMs (Jaeger's observable operator models) is to represent the state of the system as a set of predictions of observable outcomes of experiments one can do in the system. This makes PSRs rather different from history-based models such as nth-order Markov models and hidden-state-based models such as HMMs and POMDPs. We introduce an interesting construct, the system-dynamics matrix, and show how PSRs can be derived simply from it. We also use this construct to show formally that PSRs are more general than both nth-order Markov models and HMMs/POMDPs. Finally, we discuss the main difference between PSRs and OOMs and conclude with directions for future work.