The pragmatics of referring and the modality of communication
Computational Linguistics
Repairing reference identification failures by relaxation
ACL '85 Proceedings of the 23rd annual meeting on Association for Computational Linguistics
An eclectic approach to building natural language interfaces
ACL '85 Proceedings of the 23rd annual meeting on Association for Computational Linguistics
The human-computer interaction handbook
The Role of Gesture in Multimodal Referring Actions
ICMI '02 Proceedings of the 4th IEEE International Conference on Multimodal Interfaces
Natural and simulated pointing
EACL '87 Proceedings of the third conference on European chapter of the Association for Computational Linguistics
Incremental parsing and reason maintenance
COLING '90 Proceedings of the 13th conference on Computational linguistics - Volume 3
Multi-Modal Definite Clause Grammar
COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 2
COLING '92 Proceedings of the 14th conference on Computational linguistics - Volume 4
Multi-Modal-Method: a design method for building multi-modal systems
COLING '96 Proceedings of the 16th conference on Computational linguistics - Volume 2
Natural language with integrated deictic and graphic gestures
HLT '89 Proceedings of the workshop on Speech and Natural Language
Working with robots and objects: revisiting deictic reference for achieving spatial common ground
Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction
Spatial scaffolding for sociable robot learning
AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 3
Natural language and exploration of an information space: the ALFresco interactive system
IJCAI'91 Proceedings of the 12th international joint conference on Artificial intelligence - Volume 2
Generating cross-references for multimedia explanation
AAAI'92 Proceedings of the tenth national conference on Artificial intelligence
Improving gestural communication in virtual characters
AMDO'12 Proceedings of the 7th international conference on Articulated Motion and Deformable Objects
Robot deictics: how gesture and context shape referential communication
Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction
| Hi-index | 0.00 |
In virtually all current natural-language dialog systems, users can only refer to objexts by using linguistic descriptions. However, in human face-to-face conversation, participants frequently use various sorts of deictic gestures as well. In this paper, we will present the referent identification component of XTRA, a system for a natural-language access to expert systems. XTRA allows the user to combine NL input together with pointing gestures on the terminal screen in order to refer to objects on the display. Information about the location and type of this deictic gesture, as well as about the linguistic description of the referred object, the case frame, and the dialog memory are utilized for identifying the object. The system is tolerant in respect to impreciseness of both the deictic and the natural language input. The user can thereby refer to objects more easily, avoid referential failures, and employ vague everyday terms instead of precise technical notions.