Temporal difference learning and TD-Gammon
Communications of the ACM
The handbook of brain theory and neural networks
The handbook of brain theory and neural networks
Introduction to Reinforcement Learning
Introduction to Reinforcement Learning
Metalearning and neuromodulation
Neural Networks - Computational models of neuromodulation
Control of exploitation-exploration meta-parameter in reinforcement learning
Neural Networks - Computational models of neuromodulation
Reinforcement Learning in Continuous Time and Space
Neural Computation
A Computational Model of How the Basal Ganglia Produce Sequences
Journal of Cognitive Neuroscience
Optimizing dialogue management with reinforcement learning: experiments with the NJFun system
Journal of Artificial Intelligence Research
Learning to trade via direct reinforcement
IEEE Transactions on Neural Networks
An Actor-Critic based controller for glucose regulation in type 1 diabetes
Computer Methods and Programs in Biomedicine
| Hi-index | 0.01 |
We present a comprehensive model of basal ganglia in which the three important reinforcement learning components-Actor, Critic and Explorer (ACE),-are represented and their anatomical substrates are identified. Particularly, we identify the subthalamic-nucleus and globus pallidus externa (STN-GPe) loop as the Explorer, and argue that complex activity of STN and GPe neurons, found in experimental studies, provides the stochastic drive necessary for exploration. Simulations involving a two-link arm model show task-dependent variations in complexity of STN-GPe activity when the ACE network is trained to perform simple reaching movements. Complexity and average levels of STN-GPe activity are observed to be higher before training than in post-training conditions. Further, in order to simulate Parkinsonian conditions, when dopamine levels in substantia nigra portion of the model are reduced, the arm displayed, as a primary change, small amplitude movements, which on persistent network training, amplified to large amplitude unregulated movements reminiscent of Parkinsonian tremor.