A computational theory of grounding in natural language conversation
A computational theory of grounding in natural language conversation
Exploration bonuses and dual control
Machine Learning
Understanding Natural Language
Understanding Natural Language
Being Digital
Graphical Models: Foundations of Neural Computation
Graphical Models: Foundations of Neural Computation
Toward conversational human-computer interaction
AI Magazine
Information Sciences—Informatics and Computer Science: An International Journal - Special issue: Spoken language analysis, modeling and recognition-statistical and adaptive connectionist approaches
Learning Semantic Combinatoriality from the Interaction between Linguistic and Behavioral Processes
Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems
Integration of speech and vision using mutual information
ICASSP '00 Proceedings of the Acoustics, Speech, and Signal Processing, 2000. on IEEE International Conference - Volume 04
Inferring parameters and structure of latent variable models by variational bayes
UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Learning object-manipulation verbs for human-robot communication
Proceedings of the 2007 workshop on Multimodal interfaces in semantic interaction
Acquiring a Functionally Compositional System of Goal-Directed Actions of a Simulated Agent
SAB '08 Proceedings of the 10th international conference on Simulation of Adaptive Behavior: From Animals to Animats
CHI '09 Extended Abstracts on Human Factors in Computing Systems
A Concept Generation Method Based on Mutual Information Quantity among Multiple Self-organizing Maps
ICONIP '09 Proceedings of the 16th International Conference on Neural Information Processing: Part II
Learning Novel Objects for Extended Mobile Manipulation
Journal of Intelligent and Robotic Systems
Hi-index | 0.00 |
This paper describes a machine learning method that enables robots to learn the capability of linguistic communication from scratch through verbal and nonverbal interaction with users. The method focuses on two major problems that should be pursued to realize natural human-machine conversation: a scalable grounded symbol system and belief sharing. The learning is performed in the process of joint perception and joint action with a user. The method enables the robot to learn beliefs for communication by combining speech, visual, and behavioral reinforcement information in a probabilistic framework. The beliefs learned include speech units like phonemes or syllables, a lexicon, grammar, and pragmatic knowledge, and they are integrated in a system represented by a dynamical graphical model. The method also enables the user and the robot to infer the state of each other's beliefs related to communication. To facilitate such inference, the belief system held by the robot possesses a structure that represents the assumption of shared beliefs and allows for fast and robust adaptation of it through communication with the user. This adaptive behavior of the belief systems is modeled by the structural coupling of the belief systems held by the robot and the user, and it is performed through incremental online optimization in the process of interaction. Experimental results reveal that through a practical, small number of learning episodes with a user, the robot was eventually able to understand even fragmental and ambiguous utterances, act upon them, and generate utterances appropriate for the given situation. This work discusses the importance of properly handling the risk of being misunderstood in order to facilitate mutual understanding and to keep the coupling effective.