Real-time binocular smooth pursuit
International Journal of Computer Vision
Where is "it"? Event Synchronization in Gaze-Speech Input Systems
Proceedings of the 5th international conference on Multimodal interfaces
Ellipsis resolution with underspecified scope
ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Interactive vision to detect target objects for helper robots
ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
Working with robots and objects: revisiting deictic reference for achieving spatial common ground
Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction
JSAI'06 Proceedings of the 20th annual conference on New frontiers in artificial intelligence
Recognition of household objects by service robots through interactive and autonomous methods
ISVC'07 Proceedings of the 3rd international conference on Advances in visual computing - Volume Part II
Two-handed gesture recognition and fusion with speech to command a robot
Autonomous Robots
Hi-index | 0.00 |
Speech interfaces should have a capability of dealing with inexplicit utterances including such as ellipsis and deixis since they are common phenomena in our daily conversation. Their resolution using context and a priori knowledge has been investigated in the fields of natural language and speech understanding. However, there are utterances that cannot be understood by such symbol processing alone. In this paper, we consider inexplicit utterances caused from the fact that humans have vision. If we are certain that the listeners share some visual information, we often omit or mention ambiguously things about it in our utterances. We propose a method of understanding speech with such ambiguities using computer vision. It tracks the human's gaze direction, detecting objects in the direction. It also recognizes the human's actions. Based on these bits of visual information, it understands the human's inexplicit utterances. Experimental results show that the method helps to realize human-friendly speech interfaces.