Learning to Predict by the Methods of Temporal Differences
Machine Learning
Dissociating Hippocampal versus Basal Ganglia Contributions to Learning and Transfer
Journal of Cognitive Neuroscience
Using predictive representations to improve generalization in reinforcement learning
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
| Hi-index | 0.00 |
Learning when and how to generalize knowledge from past experience to novel circumstances is a challenging problem many agents face. In animals, this generalization can be caused by mediated conditioning--when two stimuli gain a relationship through the mediation of a third stimulus. For example, in sensory preconditioning, if a light is always followed by a tone, and that tone is later paired with a shock, the light will come to elicit a fear reaction, even though the light was never directly paired with shock. In this paper, we present a computational model of mediated conditioning based on reinforcement learning with predictive representations. In the model, animals learn to predict future observations through the temporal-difference algorithm. These predictions are generated using both current observations and other predictions. The model was successfully applied to a range of animal learning phenomena, including sensory preconditioning, acquired equivalence, and mediated aversion. We suggest that animals and humans are fruitfully understood as representing their world as a set of chained predictions and propose that generalization in artificial agents may benefit from a similar approach.