What's there to talk about?: a multi-modal model of referring behavior in the presence of shared visual information

Authors:
Darren Gergle
Affiliations:
Carnegie Mellon University, Pittsburg, PA
Venue:
EACL '06 Proceedings of the Eleventh Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop
Year:
2006

Citing 25
Cited 0

Attention, intentions, and the structure of discourse

Computational Linguistics
Interpretation as abduction

Artificial Intelligence - Special volume on natural language processing
Turning away from talking heads: the use of video-as-data in neurosurgery

INTERCHI '93 Proceedings of the INTERCHI '93 conference on Human factors in computing systems
Automatic referent resolution of deictic and anaphoric expressions

Computational Linguistics
Centering: a framework for modeling the local coherence of discourse

Computational Linguistics
Some advantages of video conferencing over high-quality audio conferencing: fluency and awareness of attentional focus

International Journal of Human-Computer Studies
Peripheral participation in video-mediated communication

International Journal of Human-Computer Studies
Effects of head-mounted and scene-oriented video systems on remote collaboration on physical tasks

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Cognitive Status and Form of Reference in Multimodal Human-Computer Interaction

Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence
A corpus-based evaluation of centering and pronoun resolution

Computational Linguistics - Special issue on computational anaphora resolution
Never look back: an alternative to centering

COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 2
A centering approach to pronouns

ACL '87 Proceedings of the 25th annual meeting on Association for Computational Linguistics
Linguistic theories in efficient multimodal reference resolution: an empirical investigation

Proceedings of the 10th international conference on Intelligent user interfaces
Empirical evaluations of pronoun resolution

Empirical evaluations of pronoun resolution
Centering: A Parametric Theory and Its Instantiations

Computational Linguistics
Coordination and context-dependence in the generation of embodied conversation

INLG '00 Proceedings of the first international conference on Natural language generation - Volume 14
An information-state approach to collaborative reference

ACLdemo '05 Proceedings of the ACL 2005 on Interactive poster and demonstration sessions
Two diverse systems built using generic components for spoken dialogue: (recent progress on TRIPS)

ACLdemo '05 Proceedings of the ACL 2005 on Interactive poster and demonstration sessions
Multimodal interactive maps: designing for human performance

Human-Computer Interaction
Cooperative work and shared visual context: an empirical study of comprehension problems in side-by-side and remote help dialogues

Human-Computer Interaction
Visual information as a conversational resource in collaborative physical tasks

Human-Computer Interaction
Things to talk about when talking about things

Human-Computer Interaction
Gestures over video streams to support remote collaboration on physical tasks

Human-Computer Interaction
Grounded semantic composition for visual scenes

Journal of Artificial Intelligence Research
Utilizing visual attention for cross-modal coreference interpretation

CONTEXT'05 Proceedings of the 5th international conference on Modeling and Using Context

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes the development of a rule-based computational model that describes how a feature-based representation of shared visual information combines with linguistic cues to enable effective reference resolution. This work explores a language-only model, a visual-only model, and an integrated model of reference resolution and applies them to a corpus of transcribed task-oriented spoken dialogues. Preliminary results from a corpus-based analysis suggest that integrating information from a shared visual environment can improve the performance and quality of existing discourse-based models of reference resolution.