Information fusion for visual reference resolution in dynamic situated dialogue

Authors:
Geert-Jan M. Kruijff;John D. Kelleher;Nick Hawes
Affiliations:
Language Technology Lab, DFKI GmbH;Dublin Institute of Technology;School of Computer Science, University of Birmingham
Venue:
PIT'06 Proceedings of the 2006 international tutorial and research conference on Perception and Interactive Technologies
Year:
2006

Citing 10
Cited 7

Discourse interpretation and the scope of operators

Discourse interpretation and the scope of operators
The Open Agent Architecture

Autonomous Agents and Multi-Agent Systems
An architecture for a generic dialogue shell

Natural Language Engineering
Distinctive Image Features from Scale-Invariant Keypoints

International Journal of Computer Vision
Meaningful conversation with a mobile robot

EACL '03 Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics - Volume 2
Multi-modal combinatory categorial grammar

EACL '03 Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics - Volume 1
Coupling CCG and hybrid logic dependency semantics

ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Less is more: using a single knowledge representation in dialogue systems

HLT-NAACL-TEXTMEANING '03 Proceedings of the HLT-NAACL 2003 workshop on Text meaning - Volume 9
Structural descriptions in human-assisted robot visual learning

Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction
Maintaining coherent perceptual information using anchoring

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence

Incremental generation of spatial referring expressions in situated dialog

ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
Crossmodal content binding in information-processing architectures

Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction
Applying computational models of spatial prepositions to visually situated dialog

Computational Linguistics
Towards an integrated robot with multiple cognitive functions

AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 2
Mediating between qualitative and quantitative representations for task-orientated human-robot interaction

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Tutor-based learning of visual categories using different levels of supervision

Computer Vision and Image Understanding
A cooperative scheme to aggregate spatio-temporal events in VANETs

Proceedings of the 16th International Database Engineering & Applications Sysmposium

Quantified Score

Hi-index	0.00

Visualization

Abstract

Human-Robot Interaction (HRI) invariably involves dialogue about objects in the environment in which the agents are situated. The paper focuses on the issue of resolving discourse references to such visual objects. The paper addresses the problem using strategies for intra-modal fusion (identifying that different occurrences concern the same object), and inter-modal fusion, (relating object references across different modalities). Core to these strategies are sensorimotoric coordination, and ontology-based mediation between content in different modalities. The approach has been fully implemented, and is illustrated with several working examples.