Attention, intentions, and the structure of discourse
Computational Linguistics
Turning away from talking heads: the use of video-as-data in neurosurgery
INTERCHI '93 Proceedings of the INTERCHI '93 conference on Human factors in computing systems
Automatic referent resolution of deictic and anaphoric expressions
Computational Linguistics
Centering: a framework for modeling the local coherence of discourse
Computational Linguistics
Integration and synchronization of input modes during multimodal human-computer interaction
Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
Predicting hyperarticulate speech during human-computer error resolution
Speech Communication
CSCW '02 Proceedings of the 2002 ACM conference on Computer supported cooperative work
Cognitive Status and Form of Reference in Multimodal Human-Computer Interaction
Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence
Dialogue Transcription Tools
A corpus-based evaluation of centering and pronoun resolution
Computational Linguistics - Special issue on computational anaphora resolution
Functional centering: grounding referential coherence in information structure
Computational Linguistics
Never look back: an alternative to centering
COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 2
A centering approach to pronouns
ACL '87 Proceedings of the 25th annual meeting on Association for Computational Linguistics
Providing a unified account of definite noun phrases in discourse
ACL '83 Proceedings of the 21st annual meeting on Association for Computational Linguistics
Evaluating discourse processing algorithms
ACL '89 Proceedings of the 27th annual meeting on Association for Computational Linguistics
Action as language in a shared visual space
CSCW '04 Proceedings of the 2004 ACM conference on Computer supported cooperative work
Linguistic theories in efficient multimodal reference resolution: an empirical investigation
Proceedings of the 10th international conference on Intelligent user interfaces
Empirical evaluations of pronoun resolution
Empirical evaluations of pronoun resolution
Analyzing and predicting focus of attention in remote collaborative tasks
ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
Coordination and context-dependence in the generation of embodied conversation
INLG '00 Proceedings of the first international conference on Natural language generation - Volume 14
Providing support for adaptive scripting in an on-line collaborative learning environment
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The impact of delayed visual feedback on collaborative performance
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Supporting CSCL with automatic corpus analysis technology
CSCL '05 Proceedings of th 2005 conference on Computer support for collaborative learning: learning 2005: the next 10 years!
Guest Editors' Introduction: Interacting with Digital Tabletops
IEEE Computer Graphics and Applications
An exploratory analysis of partner action and camera control in a video-mediated collaborative task
CSCW '06 Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work
An information-state approach to collaborative reference
ACLdemo '05 Proceedings of the ACL 2005 on Interactive poster and demonstration sessions
Two diverse systems built using generic components for spoken dialogue: (recent progress on TRIPS)
ACLdemo '05 Proceedings of the ACL 2005 on Interactive poster and demonstration sessions
The value of shared visual information for task-oriented collaboration
The value of shared visual information for task-oriented collaboration
Multimodal interactive maps: designing for human performance
Human-Computer Interaction
Visual information as a conversational resource in collaborative physical tasks
Human-Computer Interaction
Things to talk about when talking about things
Human-Computer Interaction
Gestures over video streams to support remote collaboration on physical tasks
Human-Computer Interaction
Utilizing visual attention for cross-modal coreference interpretation
CONTEXT'05 Proceedings of the 5th international conference on Modeling and Using Context
Evaluating the effectiveness of tutorial dialogue instruction in an exploratory learning context
ITS'06 Proceedings of the 8th international conference on Intelligent Tutoring Systems
The roles of haptic-ostensive referring expressions in cooperative, task-based human-robot dialogue
Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction
Learning to interpret utterances using dialogue history
EACL '09 Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics
INTERACT'07 Proceedings of the 11th IFIP TC 13 international conference on Human-computer interaction - Volume Part II
Focusing computational visual attention in multi-modal human-robot interaction
International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction
See what i'm saying?: using Dyadic Mobile Eye tracking to study collaborative reference
Proceedings of the ACM 2011 conference on Computer supported cooperative work
Using a game controller for relaying deictic gestures in computer-mediated communication
International Journal of Human-Computer Studies
REX-J: Japanese referring expression corpus of situated dialogs
Language Resources and Evaluation
Orchestration: tv-like mixing grammars applied to video-communication for social groups
Proceedings of the 21st ACM international conference on Multimedia
How social cues shape task coordination and communication
Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
| Hi-index | 0.01 |
A number of recent studies have demonstrated that groups benefit considerably from access to shared visual information. This is due, in part, to the communicative efficiencies provided by the shared visual context. However, a large gap exists between our current theoretical understanding and our existing models. We address this gap by developing a computational model that integrates linguistic cues with visual cues in a way that effectively models reference during tightly-coupled, task-oriented interactions. The results demonstrate that an integrated model significantly outperforms existing language-only and visual-only models. The findings can be used to inform and augment the development of conversational agents, applications that dynamically track discourse and collaborative interactions, and dialogue managers for natural language interfaces.