Mutual information as a variable to differentiate the roles of gaze in the multimodal interface

Authors:
Neil Cooke;Ao Shen
Affiliations:
University of Birmingham, Birmingham, United Kingdom;University of Birmingham, Birmingham, United Kingdom
Venue:
Proceedings of the 2010 workshop on Eye gaze in intelligent human machine interaction
Year:
2010

Citing 4
Cited 0

Assessment for automatic speech recognition II: NOISEX-92: a database and an experiment to study the effect of additive noise on speech recognition systems

Speech Communication - Special issue on speech processing in adverse conditions
Integration of speech and vision using mutual information

ICASSP '00 Proceedings of the Acoustics, Speech, and Signal Processing, 2000. on IEEE International Conference - Volume 04
What's in a gaze?: the role of eye-gaze in reference resolution in multimodal conversational interfaces

Proceedings of the 13th international conference on Intelligent user interfaces
Multimodal integration-a statistical view

IEEE Transactions on Multimedia

Quantified Score

Hi-index	0.00

Visualization

Abstract

In natural interaction, gaze assumes a variety of roles that may need distinguishing between in a gaze-contingent interface. Information from other modalities has potential to make distinction easier. In this study, Mutual Information (MI) is proposed as a variable to distinguish gaze roles by using information from both gaze and speech. A pilot experiment is conducted where different gaze behaviours are elicited from people using acoustic noise. Initial results demonstrate that MI distinguishes between gaze roles moreso than gaze characteristics alone. This work shows potential for using MI as a variable to help distinguish gaze roles in multimodal interfaces and highlights the requirement to account for acoustic noise when interpreting gaze in human-machine interactions.