Gesture improves coreference resolution

Authors:
Jacob Eisenstein;Randall Davis
Affiliations:
Massachusetts Institute of Technology, Cambridge, MA;Massachusetts Institute of Technology, Cambridge, MA
Venue:
NAACL-Short '06 Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers
Year:
2006

Citing 10
Cited 10

Mutual disambiguation of recognition errors in a multimodel architecture

Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Data mining: practical machine learning tools and techniques with Java implementations

Data mining: practical machine learning tools and techniques with Java implementations
Prosody-based automatic segmentation of speech into sentences and topics

Speech Communication - Special issue on accessing information in spoken audio
Multimodal model integration for sentence unit detection

Proceedings of the 6th international conference on Multimodal interfaces
A machine learning approach to pronoun resolution in spoken dialogue

ACL '03 Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1
Towards a model of face-to-face grounding

ACL '03 Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1
Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms

EMNLP '02 Proceedings of the ACL-02 conference on Empirical methods in natural language processing - Volume 10
Correlation clustering in general weighted graphs

Theoretical Computer Science - Approximation and online algorithms
On coreference resolution performance metrics

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Gesture features for coreference resolution

MLMI'06 Proceedings of the Third international conference on Machine Learning for Multimodal Interaction

Semantic back-pointers from gesture

NAACL-DocConsortium '06 Proceedings of the 2006 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: companion volume: doctoral consortium
Sketch understanding systems

ACM SIGGRAPH 2007 courses
Turning lectures into comic books using linguistically salient gestures

AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 1
Gesture salience as a hidden variable for coreference resolution and keyframe extraction

Journal of Artificial Intelligence Research
Communicative gestures in coreference identification in multiparty meetings

Proceedings of the 2009 international conference on Multimodal interfaces
Hand gestures in disambiguating types of you expressions in multiparty meetings

SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Utilizing gestures to improve sentence boundary detection

Multimedia Tools and Applications
Improving pronominal and deictic co-reference resolution with multi-modal features

SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
Co-reference via pointing and haptics in multi-modal dialogues

NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Speech and gesture interaction in an Ambient assisted living lab

SMIAE '12 Proceedings of the 1st Workshop on Speech and Multimodal Interaction in Assistive Environments

Quantified Score

Hi-index	0.00

Visualization

Abstract

Coreference resolution, like many problems in natural language processing, has most often been explored using datasets of written text. While spontaneous spoken language poses well-known challenges, it also offers additional modalities that may help disambiguate some of the inherent disfluency. We explore features of hand gesture that are correlated with coreference. Combining these features with a traditional textual model yields a statistically significant improvement in overall performance.