A possibility for implementing curiosity and boredom in model-building neural controllers
Proceedings of the first international conference on simulation of adaptive behavior on From animals to animats
Dopamine-dependent plasticity of corticostriatal synapses
Neural Networks - Computational models of neuromodulation
Actor-critic models of the basal ganglia: new anatomical and computational perspectives
Neural Networks - Computational models of neuromodulation
Dopamine: generalization and bonuses
Neural Networks - Computational models of neuromodulation
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Evolution and learning in an intrinsically motivated reinforcement learning robot
ECAL'07 Proceedings of the 9th European conference on Advances in artificial life
Tutorial on Neural Systems Modeling
Tutorial on Neural Systems Modeling
The roles of the amygdala in the affective regulation of body, brain, and behaviour
Connection Science - Affective Robotics
Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010)
IEEE Transactions on Autonomous Mental Development
Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective
IEEE Transactions on Autonomous Mental Development
Intrinsic Motivation Systems for Autonomous Mental Development
IEEE Transactions on Evolutionary Computation
Learning Generalizable Control Programs
IEEE Transactions on Autonomous Mental Development
Hi-index | 0.00 |
Reinforcement (trial-and-error) learning in animals is driven by a multitude of processes. Most animals have evolved several sophisticated systems of 'extrinsic motivations' (EMs) that guide them to acquire behaviours allowing them to maintain their bodies, defend against threat, and reproduce. Animals have also evolved various systems of 'intrinsic motivations' (IMs) that allow them to acquire actions in the absence of extrinsic rewards. These actions are used later to pursue such rewards when they become available. Intrinsic motivations have been studied in Psychology for many decades and their biological substrates are now being elucidated by neuroscientists. In the last two decades, investigators in computational modelling, robotics and machine learning have proposed various mechanisms that capture certain aspects of IMs. However, we still lack models of IMs that attempt to integrate all key aspects of intrinsically motivated learning and behaviour while taking into account the relevant neurobiological constraints. This paper proposes a bio-constrained system-level model that contributes a major step towards this integration. The model focusses on three processes related to IMs and on the neural mechanisms underlying them: (a) the acquisition of action-outcome associations (internal models of the agent-environment interaction) driven by phasic dopamine signals caused by sudden, unexpected changes in the environment; (b) the transient focussing of visual gaze and actions on salient portions of the environment; (c) the subsequent recall of actions to pursue extrinsic rewards based on goal-directed reactivation of the representations of their outcomes. The tests of the model, including a series of selective lesions, show how the focussing processes lead to a faster learning of action-outcome associations, and how these associations can be recruited for accomplishing goal-directed behaviours. The model, together with the background knowledge reviewed in the paper, represents a framework that can be used to guide the design and interpretation of empirical experiments on IMs, and to computationally validate and further develop theories on them.