Elements of information theory
Elements of information theory
Feature extraction by non parametric mutual information maximization
The Journal of Machine Learning Research
Learning and discovery of predictive state representations in dynamical systems with reset
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Predictive state representations: a new theory for modeling dynamical systems
UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Learning predictive representations from a history
ICML '05 Proceedings of the 22nd international conference on Machine learning
Learning predictive state representations in dynamical systems without reset
ICML '05 Proceedings of the 22nd international conference on Machine learning
Predictive linear-Gaussian models of controlled stochastic dynamical systems
ICML '06 Proceedings of the 23rd international conference on Machine learning
Kernel Predictive Linear Gaussian models for nonlinear stochastic dynamical systems
ICML '06 Proceedings of the 23rd international conference on Machine learning
Mixtures of predictive linear Gaussian models for nonlinear stochastic dynamical systems
AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Relational knowledge with predictive state representations
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Using predictive representations to improve generalization in reinforcement learning
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Closing the learning-planning loop with predictive state representations
International Journal of Robotics Research
Construction of approximation spaces for reinforcement learning
The Journal of Machine Learning Research
| Hi-index | 0.00 |
Models of agent-environment interaction that use predictive state representations (PSRs) have mainly focused on the case of discrete observations and actions. The theory of discrete PSRs uses an elegant construct called the system dynamics matrix and derives the notion of predictive state as a sufficient statistic via the rank of the matrix. With continuous observations and actions, such a matrix and its rank no longer exist. In this paper, we show how to define an analogous construct for the continuous case, called the system dynamics distributions, and use information theoretic notions to define a sufficient statistic and thus state. Given this new construct, we use kernel density estimation to learn approximate system dynamics distributions from data, and use information-theoretic tools to derive algorithms for discovery of state and learning of model parameters. We illustrate our new modeling method on two example problems.