Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning
Artificial Intelligence
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Metalearning and neuromodulation
Neural Networks - Computational models of neuromodulation
A distributed representation of temporal context
Journal of Mathematical Psychology
Learning to Predict by the Methods of Temporal Differences
Machine Learning
2005 Special issue: Interpreting hippocampal function as recoding and forecasting
Neural Networks - Special issue: Computational theories of the functions of the hippocampus
The Journal of Machine Learning Research
Hi-index | 0.00 |
The successor representation was introduced into reinforcement learning by Dayan (1993) as a means of facilitating generalization between states with similar successors. Although reinforcement learning in general has been used extensively as a model of psychological and neural processes, the psychological validity of the successor representation has yet to be explored. An interesting possibility is that the successor representation can be used not only for reinforcement learning but for episodic learning as well. Our main contribution is to show that a variant of the temporal context model (TCM; Howard & Kahana, 2002), an influential model of episodic memory, can be understood as directly estimating the successor representation using the temporal difference learning algorithm (Sutton & Barto, 1998). This insight leads to a generalization of TCM and new experimental predictions. In addition to casting a new normative light on TCM, this equivalence suggests a previously unexplored point of contact between different learning systems.