Learning movement sequences with a delayed reward signal in a hierarchical model of motor function

Authors:
S. M. Stringer;E. T. Rolls;P. Taylor
Affiliations:
Oxford University, Centre for Computational Neuroscience, Department of Experimental Psychology, South Parks Road, Oxford OX1 3UD, United Kingdom;Oxford University, Centre for Computational Neuroscience, Department of Experimental Psychology, South Parks Road, Oxford OX1 3UD, United Kingdom;Oxford University, Centre for Computational Neuroscience, Department of Experimental Psychology, South Parks Road, Oxford OX1 3UD, United Kingdom
Venue:
Neural Networks
Year:
2007

Citing 4
Cited 2

Forward models for physiological motor control

Neural Networks - 1996 Special issue: four major hypotheses in neuroscience
Learning to Predict by the Methods of Temporal Differences

Machine Learning
Self-organizing continuous attractor networks and motor function

Neural Networks
2005 Special issue: Spatial view cells in the hippocampus, and their idiothetic update based on place and head direction

Neural Networks - Special issue: Computational theories of the functions of the hippocampus

Hierarchical dynamical models of motor function

Neurocomputing
An embodied account of serial order: How instabilities drive sequence generation

Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

A key problem in reinforcement learning is how an animal is able to learn a sequence of movements when the reward signal only occurs at the end of the sequence. We describe how a hierarchical dynamical model of motor function is able to solve the problem of delayed reward in learning movement sequences using associative (Hebbian) learning. At the lowest level, the motor system encodes simple movements or primitives, while at higher levels the system encodes sequences of primitives. During training, the network is able to learn a high level motor program composed of a specific temporal sequence of motor primitives. The network is able to achieve this despite the fact that the reward signal, which indicates whether or not the desired motor program has been performed correctly, is received only at the end of each trial during learning. Use of a continuous attractor network in the architecture enables the network to generate the motor outputs required to produce the continuous movements necessary to implement the motor sequence.