Closing the Sensory-Motor Loop on Dopamine Signalled Reinforcement Learning

Authors:
Paul Chorley;Anil K. Seth
Affiliations:
Dept of Informatics, University of Sussex, Brighton, UK BN1 9QJ;Dept of Informatics, University of Sussex, Brighton, UK BN1 9QJ
Venue:
SAB '08 Proceedings of the 10th international conference on Simulation of Adaptive Behavior: From Animals to Animats
Year:
2008

Citing 5
Cited 0

Evolving action selection and selective attention without actions, attention, or selection

Proceedings of the fifth international conference on simulation of adaptive behavior on From animals to animats 5
Spikes: exploring the neural code

Spikes: exploring the neural code
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex

Neural Computation
Simple model of spiking neurons

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

It has been shown recently that dopamine signalled modulation of spike timing-dependent synaptic plasticity (DA-STDP) can enable reinforcement learning of delayed stimulus-reward associations when both stimulus and reward are delivered at precisely timed intervals. Here, we test whether a similar model can support learning in an embodied context, in which timing of both sensory input and delivery of reward depend on the agent's behaviour. We show that effective reinforcement learning is indeed possible, but only when stimuli are gated so as to occur as near-synchronous patterns of neural activity and when neuroanatomical constraints are imposed which predispose agents to exploratative behaviours. Extinction of learned responses in this model is subsequently shown to result from agent-environment interactions and not directly from any specific neural mechanism.