A drive-reinforcement model of single neuron function: An alternative to the Hebbian neuronal model
AIP Conference Proceedings 151 on Neural Networks for Computing
AIP Conference Proceedings 151 on Neural Networks for Computing
Practical Issues in Temporal Difference Learning
Machine Learning
Technical Note: \cal Q-Learning
Machine Learning
The Convergence of TD(λ) for General λ
Machine Learning
TD(λ) Converges with Probability 1
Machine Learning
Reinforcement learning with replacing eligibility traces
Machine Learning - Special issue on reinforcement learning
Computational models of classical conditioning: a comparative study
Proceedings of the fifth international conference on simulation of adaptive behavior on From animals to animats 5
Machine Learning
Learning continuous attractors in recurrent networks
NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Temporally asymmetric Hebbian learning, spike timing and neuronal response variability
Proceedings of the 1998 conference on Advances in neural information processing systems II
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Dopamine-dependent plasticity of corticostriatal synapses
Neural Networks - Computational models of neuromodulation
Actor-critic models of the basal ganglia: new anatomical and computational perspectives
Neural Networks - Computational models of neuromodulation
Learning to Predict by the Methods of Temporal Differences
Machine Learning
Spike-Timing Dependent Competitive Learning of Integrate-and-Fire Neurons with Active Dendrites
ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Does Morphology Influence Temporal Plasticity?
ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Isotropic sequence order learning
Neural Computation
Neural Computation
Dynamic Programming
Temporal credit assignment in reinforcement learning
Temporal credit assignment in reinforcement learning
Memory encoding by theta phase precession in the hippocampal network
Neural Computation
Reinforcement learning models of the dopamine system and their behavioral implications
Reinforcement learning models of the dopamine system and their behavioral implications
Temporal Difference Model Reproduces Anticipatory Neural Activity
Neural Computation
Intrinsic Stabilization of Output Rates by Spike-Based Hebbian Learning
Neural Computation
Spike-Timing-Dependent Hebbian Plasticity as Temporal Difference Learning
Neural Computation
Reinforcement Learning in Continuous Time and Space
Neural Computation
A Computational Model of How the Basal Ganglia Produce Sequences
Journal of Cognitive Neuroscience
The hippocampus and cerebellum in adaptively timed learning, recognition, and movement
Journal of Cognitive Neuroscience
On the convergence of stochastic iterative dynamic programming algorithms
Neural Computation
Reinforcement learning: a survey
Journal of Artificial Intelligence Research
Elman Backpropagation as Reinforcement for Simple Recurrent Networks
Neural Computation
PRESENCE: A Human-Inspired Architecture for Speech-Based Human-Machine Interaction
IEEE Transactions on Computers
Modeling dopamine activity by Reinforcement Learning methods: implications from two recent models
Artificial Intelligence Review
A Simple Aplysia-Like Spiking Neural Network to Generate Adaptive Behavior in Autonomous Robots
Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
Design Principles and Constraints Underlying the Construction of Brain-Based Devices
Neural Information Processing
A spiking neural network model of an actor-critic learning agent
Neural Computation
On the relation between bursts and dynamic synapse properties: a modulation-based Ansatz
Computational Intelligence and Neuroscience
Development of Symbiotic Brain-Machine Interfaces Using a Neurophysiology Cyberworkstation
Proceedings of the 13th International Conference on Human-Computer Interaction. Part II: Novel Interaction Methods and Techniques
Learning anticipation via spiking networks: application to navigation control
IEEE Transactions on Neural Networks
Spiking neuron model for temporal sequence recognition
Neural Computation
Prerequesites for symbiotic brain-machine interfaces
SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Biasing neural networks towards exploration or exploitation using neuromodulation
ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
The neuronal replicator hypothesis
Neural Computation
Back-propagation as reinforcement in prediction tasks
ICANN'05 Proceedings of the 15th international conference on Artificial neural networks: formal models and their applications - Volume Part II
Stabilising hebbian learning with a third factor in a food retrieval task
SAB'06 Proceedings of the 9th international conference on From Animals to Animats: simulation of Adaptive Behavior
Toward nonlinear local reinforcement learning rules through neuroevolution
Neural Computation
| Hi-index | 0.00 |
In this review, we compare methods for temporal sequence learning (TSL) across the disciplines machine-control, classical conditioning, neuronal models for TSL as well as spike-timing-dependent plasticity (STDP). This review introduces the most influential models and focuses on two questions: To what degree are reward-based (e.g., TD learning) and correlationbased (Hebbian) learning related? and How do the different models correspond to possibly underlying biological mechanisms of synaptic plasticity? We first compare the different models in an open-loop condition, where behavioral feedback does not alter the learning. Here we observe that reward-based and correlation-based learning are indeed very similar. Machine control is then used to introduce the problem of closed-loop control (e.g., actor-critic architectures). Here the problem of evaluative (rewards) versus nonevaluative (correlations) feedback from the environment will be discussed, showing that both learning approaches are fundamentally different in the closed-loop condition. In trying to answer the second question, we compare neuronal versions of the different learning architectures to the anatomy of the involved brain structures (basal-ganglia, thalamus, and cortex) and the molecular biophysics of glutamatergic and dopaminergic synapses. Finally, we discuss the different algorithms used to model STDP and compare them to reward-based learning rules. Certain similarities are found in spite of the strongly different timescales. Here we focus on the biophysics of the different calciumrelease mechanisms known to be involved in STDP.