Improving pronominal and deictic co-reference resolution with multi-modal features

Authors:
Lin Chen;Anruo Wang;Barbara Di Eugenio
Affiliations:
University of Illinois at Chicago, Chicago, IL;University of Illinois at Chicago, Chicago, IL;University of Illinois at Chicago, Chicago, IL
Venue:
SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
Year:
2011

Citing 8
Cited 3

Assessing agreement on classification tasks: the kappa statistic

Computational Linguistics
Cognitive Status and Form of Reference in Multimodal Human-Computer Interaction

Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence
A machine learning approach to pronoun resolution in spoken dialogue

ACL '03 Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1
On coreference resolution performance metrics

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Beyond attention: the role of deictic gesture in intention recognition in multimodal conversational interfaces

Proceedings of the 13th international conference on Intelligent user interfaces
Gesture improves coreference resolution

NAACL-Short '06 Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers
The WEKA data mining software: an update

ACM SIGKDD Explorations Newsletter
Utilizing gestures to improve sentence boundary detection

Multimedia Tools and Applications

Co-reference via pointing and haptics in multi-modal dialogues

NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Improving sentence completion in dialogues with multi-modal features

SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Speech and gesture interaction in an Ambient assisted living lab

SMIAE '12 Proceedings of the 1st Workshop on Speech and Multimodal Interaction in Assistive Environments

Quantified Score

Hi-index	0.00

Visualization

Abstract

Within our ongoing effort to develop a computational model to understand multi-modal human dialogue in the field of elderly care, this paper focuses on pronominal and deictic co-reference resolution. After describing our data collection effort, we discuss our annotation scheme. We developed a co-reference model that employs both a simple notion of markable type, and multiple statistical models. Our results show that knowing the type of the markable, and the presence of simultaneous pointing gestures improve co-reference resolution for personal and deictic pronouns.