Temporal Difference Model Reproduces Anticipatory Neural Activity

Authors:
Roland E. Suri;Wolfram E. Schultz
Affiliations:
Institut de Physiologie, 1700 Fribourg, Switzerland;Institut de Physiologie, 1700 Fribourg, Switzerland
Venue:
Neural Computation
Year:
2001

Citing 5
Cited 22

Adaptive timing in neural networks: the conditioned response

Biological Cybernetics
Neural dynamics of adaptive timing temporal discrimination during associative learning

Neural Networks
Dynamic model of visual recognition predicts neural response properties in the visual cortex

Neural Computation
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
A model of corticostriatal plasticity for learning oculomotor associations and sequences

Journal of Cognitive Neuroscience

TD Models of reward predictive responses in dopamine neurons

Neural Networks - Computational models of neuromodulation
An integrative theory of the phasic and tonic modes of dopamine modulation in the prefrontal cortex

Neural Networks - Computational models of neuromodulation
Neuromodulation and Plasticity in an autonomous robot

Neural Networks - Computational models of neuromodulation
Timed delivery of reward signals in an autonomous robot

ICSAB Proceedings of the seventh international conference on simulation of adaptive behavior on From animals to animats
Isotropic sequence order learning

Neural Computation
A generic architecture for adaptive agents based on reinforcement learning

Information Sciences—Informatics and Computer Science: An International Journal - Special issue: Bio-inspired systems (BIS)
Actor-Critic Models of Reinforcement Learning in the Basal Ganglia: From Natural to Artificial Rats

Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
Temporal Sequence Learning, Prediction, and Control: A Review of Different Models and Their Relation to Biological Mechanisms

Neural Computation
Attention-Gated Reinforcement Learning of Internal Representations for Classification

Neural Computation
Strongly improved stability and faster convergence of temporal sequence learning by using input correlations only

Neural Computation
Strongly improved stability and faster convergence of temporal sequence learning by using input correlations only

Neural Computation
A Model of Prefrontal Cortex Dopaminergic Modulation during the Delayed Alternation Task

Journal of Cognitive Neuroscience
Modeling embodied visual behaviors

ACM Transactions on Applied Perception (TAP)
Learning with “Relevance”: Using a Third Factor to Stabilize Hebbian Learning

Neural Computation
On the Role of Dopamine in Cognitive Vision

Attention in Cognitive Systems. Theories and Systems from an Interdisciplinary Viewpoint
A spiking neural network model of an actor-critic learning agent

Neural Computation
Brain networks involved in early versus late response anticipation and their relation to conflict processing

Journal of Cognitive Neuroscience
Architecture of behavior-based and robotics self-optimizing memory controller

ICRA'09 Proceedings of the 2009 IEEE international conference on Robotics and Automation
A computational study of pre-synaptic re-uptake of dopamine on phosphorylation of DARPP-32

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Hyperbolically discounted temporal difference learning

Neural Computation
Internal-time temporal difference model for neural value-based decision making

Neural Computation
Modeling the brain's operating system

BVAI'05 Proceedings of the First international conference on Brain, Vision, and Artificial Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

Anticipatory neural activity preceding behaviorally important events has been reported in cortex, striatum, and midbrain dopamine neurons. Whereas dopamine neurons are phasically activated by reward-predictive stimuli, anticipatory activity of cortical and striatal neurons is increased during delay periods before important events. Characteristics of dopamine neuron activity resemble those of the prediction error signal of the temporal difference (TD) model of Pavlovian learning (Sutton & Barto, 1990). This study demonstrates that the prediction signal of the TD model reproduces characteristics of cortical and striatal anticipatory neural activity. This finding suggests that tonic anticipatory activities may reflect prediction signals that are involved in the processing of dopamine neuron activity.