Learning epistemic actions in model-free memory-free reinforcement learning: experiments with a neuro-robotic model

Authors:
Dimitri Ognibene;Nicola Catenacci Volpi;Giovanni Pezzulo;Gianluca Baldassare
Affiliations:
Personal Robotics Laboratory, Imperial College London, UK;IMT Institute for Advanced Studies, Lucca, Italy;Istituto di Scienze e Tecnologie della Cognizione, CNR, Italy and Istituto di Linguistica Computazionale "Antonio Zampolli", CNR, Italy;Istituto di Scienze e Tecnologie della Cognizione, CNR, Italy
Venue:
Living Machines'13 Proceedings of the Second international conference on Biomimetic and Biohybrid Systems
Year:
2013

Citing 14
Cited 0

Integrated architecture for learning, planning, and reacting based on approximating dynamic programming

Proceedings of the seventh international conference (1990) on Machine learning
Learning to Perceive and Act by Trial and Error

Machine Learning
Reinforcement learning of non-Markov decision processes

Artificial Intelligence - Special volume on computational research on interaction and agency, part 2
The effect of representation and knowledge on goal-directed exploration with reinforcement-learning algorithms

Machine Learning - Special issue on reinforcement learning
Information Theoretic Sensor Data Selection for Active Object Recognition and State Estimation

IEEE Transactions on Pattern Analysis and Machine Intelligence
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
Scalable Internal-State Policy-Gradient Methods for POMDPs

ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping

ICML '99 Proceedings of the Sixteenth International Conference on Machine Learning
Exact and approximate algorithms for partially observable markov decision processes

Exact and approximate algorithms for partially observable markov decision processes
Model-free reinforcement learning as mixture learning

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Action representation and partially observable planning using epistemic logic

IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Thinking with external representations

AI & Society
Learning to look in different environments: an active-vision model which learns and readapts visual routines

SAB'10 Proceedings of the 11th international conference on Simulation of adaptive behavior: from animals to animats
Intrinsically Motivated Learning in Natural and Artificial Systems

Intrinsically Motivated Learning in Natural and Artificial Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Passive sensory processing is often insufficient to guide biological organisms in complex environments. Rather, behaviourally relevant information can be accessed by performing so-called epistemicactions that explicitly aim at unveiling hidden information. However, it is still unclear how an autonomous agent can learn epistemic actions and how it can use them adaptively. In this work, we propose a definition of epistemic actions for POMDPs that derive from their characterizations in cognitive science and classical planning literature. We give theoretical insights about how partial observability and epistemic actions can affect the learning process and performance in the extreme conditions of model-free and memory-free reinforcement learning where hidden information cannot be represented. We finally investigate these concepts using an integrated eye-arm neural architecture for robot control, which can use its effectors to execute epistemic actions and can exploit the actively gathered information to efficiently accomplish a seek-and-reach task.