Hand gestures in disambiguating types of you expressions in multiparty meetings

Authors:
Tyler Baldwin;Joyce Y. Chai;Katrin Kirchhoff
Affiliations:
Michigan State University, East Lansing, MI;Michigan State University, East Lansing, MI;University of Washington, Seattle, WA
Venue:
SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Year:
2010

Citing 7
Cited 0

Integration and synchronization of input modes during multimodal human-computer interaction

Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
Generating usable formats for metadata and annotations in a large meeting corpus

ACL '07 Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions
Disambiguating between generic and referential "you" in dialog

ACL '07 Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions
Who is "you"?: combining linguistic and gaze features to resolve second-person references in dialogue

EACL '09 Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics
Gesture improves coreference resolution

NAACL-Short '06 Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers
Communicative gestures in coreference identification in multiparty meetings

Proceedings of the 2009 international conference on Multimodal interfaces
Cascaded lexicalised classifiers for second-person reference resolution

SIGDIAL '09 Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue

Quantified Score

Hi-index	0.00

Visualization

Abstract

The second person pronoun you serves different functions in English. Each of these different types often corresponds to a different term when translated into another language. Correctly identifying different types of you can be beneficial to machine translation systems. To address this issue, we investigate disambiguation of different types of you occurrences in multiparty meetings with a new focus on the role of hand gesture. Our empirical results have shown that incorporation of gesture improves performance on differentiating between the generic use of you (e.g., refer to people in general) and the referential use of you (e.g., refer to a specific person or a group of people). Incorporation of gesture can also compensate for limitations in automated language processing (e.g., reliable recognition of dialogue acts) and achieve comparable results.