Relational knowledge with predictive state representations

Authors:
David Wingate;Vishal Soni;Britton Wolfe;Satinder Singh
Affiliations:
Computer Science and Engineering, University of Michigan, Ann Arbor, MI;Computer Science and Engineering, University of Michigan, Ann Arbor, MI;Computer Science and Engineering, University of Michigan, Ann Arbor, MI;Computer Science and Engineering, University of Michigan, Ann Arbor, MI
Venue:
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Year:
2007

Citing 7
Cited 3

Relational reinforcement learning

Machine Learning - Special issue on inducive logic programming
Theoretical Results on Reinforcement Learning with Temporally Abstract Options

ECML '98 Proceedings of the 10th European Conference on Machine Learning
Learning and discovery of predictive state representations in dynamical systems with reset

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Learning predictive representations from a history

ICML '05 Proceedings of the 22nd international conference on Machine learning
Markov logic networks

Machine Learning
Using predictive representations to improve generalization in reinforcement learning

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
BLOG: probabilistic models with unknown objects

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence

On discovery and learning of models with predictive representations of state for agents with continuous actions and observations

Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems
Approximate predictive state representations

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
Learning to make predictions in partially observable environments without a generative model

Journal of Artificial Intelligence Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

Most work on Predictive Representations of State (PSRs) has focused on learning and planning in unstructured domains (for example, those represented by flat POMDPs). This paper extends PSRs to represent relational knowledge about domains, so that they can use policies that generalize across different tasks, capture knowledge that ignores irrelevant attributes of objects, and represent policies in a way that is independent of the size of the state space. Using a blocks world domain, we show how generalized predictions about the future can compactly capture relations between objects, which in turn can be used to naturally specify relational-style options and policies. Because our representation is expressed solely in terms of actions and observations, it has extensive semantics which are statistics about observable quantities.