Integrative learning between language and action: a neuro-robotics experiment

Authors:
Hiroaki Arie;Tetsuro Endo;Sungmoon Jeong;Minho Lee;Shigeki Sugano;Jun Tani
Affiliations:
Brain Science Institute, RIKEN;Department of Modern Mechanical Engineering, Waseda University;School of Electrical Engineering and Computer Science, Kyungpook National University;School of Electrical Engineering and Computer Science, Kyungpook National University;Department of Modern Mechanical Engineering, Waseda University;Brain Science Institute, RIKEN
Venue:
ICANN'10 Proceedings of the 20th international conference on Artificial neural networks: Part II
Year:
2010

Citing 5
Cited 1

Learning Semantic Combinatoriality from the Interaction between Linguistic and Behavioral Processes

Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
Listening to Action-related Sentences Activates Fronto-parietal Motor Circuits

Journal of Cognitive Neuroscience
Learning Multiple Goal-Directed Actions Through Self-Organization of a Dynamic Neural Network Model: A Humanoid Robot Experiment

Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
2008 Special Issue: Achieving "organic compositionality" through self-organization: Reviews on brain-inspired robotics experiments

Neural Networks
Emergence of hierarchical structure mirroring linguistic composition in a recurrent neural network

Neural Networks

Simultaneously emerging Braitenberg codes and compositionality

Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper introduces a model for associative learning combining both linguistic and behavior modalities. The model consists of language and behavior modules both implemented by a hierarchical dynamic network model and interacting densely through hub-like neurons, the so-called parametric biases (PB). By implementing this model for a humanoid robot with the task of manipulating multiple objects, the robot was tutored to associate sentences of two different grammatical types with corresponding sensory-motor schemata. The first type was a verb followed by an objective noun such as "hold red" or "hit blue"; the second was a verb followed by an objective noun and further followed by an adverb phrase such as "Put red on blue". Our analysis of the results of a learning experiment showed that two clusters corresponding to these two types of grammatical sentences appear in the PB activity space, such that a specific micro structure is organized for each cluster.