2005 Special issue: Hippocampal replay contributes to within session learning in a temporal difference reinforcement learning model

Authors:
Adam Johnson;A. David Redish
Affiliations:
Center for Cognitive Sciences and Graduate Program in Neuroscience, University of Minnesota, MN 55455, USA;Department of Neuroscience, University of Minnesota, MN 55455, USA
Venue:
Neural Networks - Special issue: Computational theories of the functions of the hippocampus
Year:
2005

Citing 5
Cited 0

Integrated architecture for learning, planning, and reacting based on approximating dynamic programming

Proceedings of the seventh international conference (1990) on Machine learning
The role of the hippocampus in solving the Morris water maze

Neural Computation
Reinforcement learning models of the dopamine system and their behavioral implications

Reinforcement learning models of the dopamine system and their behavioral implications
Latent Attractors: A Model for Context-Dependent Place Representations in the Hippocampus

Neural Computation
A model of spatial map formation in the hippocampus of the rat

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

Temporal difference reinforcement learning (TDRL) algorithms, hypothesized to partially explain basal ganglia functionality, learn more slowly than real animals. Modified TDRL algorithms (e.g. the Dyna-Q family) learn faster than standard TDRL by practicing experienced sequences offline. We suggest that the replay phenomenon, in which ensembles of hippocampal neurons replay previously experienced firing sequences during subsequent rest and sleep, may provide practice sequences to improve the speed of TDRL learning, even within a single session. We test the plausibility of this hypothesis in a computational model of a multiple-T choice-task. Rats show two learning rates on this task: a fast decrease in errors and a slow development of a stereotyped path. Adding developing replay to the model accelerates learning the correct path, but slows down the stereotyping of that path. These models provide testable predictions relating the effects of hippocampal inactivation as well as hippocampal replay on this task.