Identifying the intended addressee in mixed human-human and human-computer interaction from non-verbal features

Authors:
Koen van Turnhout;Jacques Terken;Ilse Bakx;Berry Eggen
Affiliations:
Eindhoven University of Technology, The Netherlands;Eindhoven University of Technology, The Netherlands;Eindhoven University of Technology, The Netherlands;Eindhoven University of Technology, The Netherlands
Venue:
ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
Year:
2005

Citing 5
Cited 18

Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Video cut editing rule based on participants' gaze in multiparty conversation

MULTIMEDIA '03 Proceedings of the eleventh ACM international conference on Multimedia
Providing the basis for human-robot-interaction: a multi-modal attention system for a mobile robot

Proceedings of the 5th international conference on Multimodal interfaces
A multi-modal approach for determining speaker location and focus

Proceedings of the 5th international conference on Multimodal interfaces
Identifying the addressee in human-human-robot interactions based on head pose and speech

Proceedings of the 6th international conference on Multimodal interfaces

Human-centered collaborative interaction

Proceedings of the 1st ACM international workshop on Human-centered multimedia
Human perception of intended addressee during computer-assisted meetings

Proceedings of the 8th international conference on Multimodal interfaces
Toward open-microphone engagement for multiparty interactions

Proceedings of the 8th international conference on Multimodal interfaces
Tracking head pose and focus of attention with multiple far-field cameras

Proceedings of the 8th international conference on Multimodal interfaces
Extraction of important interactions in medical interviewsusing nonverbal information

Proceedings of the 9th international conference on Multimodal interfaces
Multimodalcues for addressee-hood in triadic communication with a human information retrieval agent

Proceedings of the 9th international conference on Multimodal interfaces
Implicit user-adaptive system engagement in speech and pen interfaces

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces

HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
Designing Socially Aware Conversational Agents

PIT '08 Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems
Who is "you"?: combining linguistic and gaze features to resolve second-person references in dialogue

EACL '09 Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics
Are you being addressed?: real-time addressee detection to support remote participants in hybrid meetings

SIGDIAL '09 Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Cascaded lexicalised classifiers for second-person reference resolution

SIGDIAL '09 Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Intuition as instinctive dialogue

Computing with instinct
Learning speaker, addressee and overlap detection models from multimodal streams

Proceedings of the 14th ACM international conference on Multimodal interaction
Given that, should i respond?: contextual addressee estimation in multi-party human-robot interactions

Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction
Implementation and evaluation of a multimodal addressee identification mechanism for multiparty conversation systems

Proceedings of the 15th ACM on International conference on multimodal interaction
Leveraging the robot dialog state for visual focus of attention recognition

Proceedings of the 15th ACM on International conference on multimodal interaction
Context aware addressee estimation for human robot interaction

Proceedings of the 6th workshop on Eye gaze in intelligent human machine interaction: gaze in multimodal interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

Against the background of developments in the area of speech-based and multimodal interfaces, we present research on determining the addressee of an utterance in the context of mixed human-human and multimodal human-computer interaction. Working with data that are taken from realistic scenarios, we explore several features with respect to their relevance to the question who is the addressee of an utterance: eye gaze both of speaker and listener, dialogue history and utterance length. With respect to eye gaze, we inspect the detailed timing of shifts in eye gaze between different communication partners (human or computer). We show that these features result in an improved classification of utterances in terms of addressee-hood relative to a simple classification algorithm that assumes that "the addressee is where the eye is", and compare our results to alternative approaches.