Probabilistic inference of gaze patterns and structure of multiparty conversations from head directions and utterances

Authors:
Kazuhiro Otsuka;Yoshinao Takemae;Junji Yamato;Hiroshi Murase
Affiliations:
NTT Communication Science Laboratories, Nippon Telegraph and Telephone Corporation, Atsugi, Japan;NTT Cyber Solutions Laboratories, Nippon Telegraph and Telephone Corporation, Yokosuka, Japan;NTT Communication Science Laboratories, Nippon Telegraph and Telephone Corporation, Atsugi, Japan;Graduate School of Information Science, Nagoya University, Nagoya, Japan
Venue:
JSAI'05 Proceedings of the 2005 international conference on New Frontiers in Artificial Intelligence
Year:
2006

Citing 9
Cited 1

A Bayesian Computer Vision System for Modeling Human Interactions

IEEE Transactions on Pattern Analysis and Machine Intelligence
Distributed meetings: a meeting capture and broadcasting system

Proceedings of the tenth ACM international conference on Multimedia
An Algorithm for Real-Time Stereo Vision Implementation of Head Pose and Gaze Direction Measurement

FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
A free-head, simple calibration, gaze tracking system that enables gaze-based interaction

Proceedings of the 2004 symposium on Eye tracking research & applications
Impact of video editing based on participants' gaze in multiparty conversation

CHI '04 Extended Abstracts on Human Factors in Computing Systems
Automatic Analysis of Multimodal Group Actions in Meetings

IEEE Transactions on Pattern Analysis and Machine Intelligence
Adaptive view-based appearance models

CVPR'03 Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition
Blind restoration of linearly degraded discrete signals by Gibbssampling

IEEE Transactions on Signal Processing
Modeling focus of attention for meeting indexing based on multiple cues

IEEE Transactions on Neural Networks

Estimating human body and head orientation change to detect visual attention direction

ACCV'10 Proceedings of the 2010 international conference on Computer vision - Volume Part I

Quantified Score

Hi-index	0.00

Visualization

Abstract

A novel probabilistic framework is proposed for inferring gaze patterns and the structure of conversation in face-to-face multiparty communication, based on head directions and the presence/absence of utterances of participants. First, we define three classes of conversational regimes, which are characterized by the topology of the gaze pattern; we assume that they indicate the structure of the conversation, i.e. who is talking to whom. Next, the problem is formulated as joint estimation of both regime state from the gaze pattern and utterance, and the gaze pattern from head directions. We then devise a dynamic Bayesian network, called the Markov-switching model. The regime changes over time are based on Markov transitions, and controls the dynamics of the gaze patterns and utterances. Furthermore, Bayesian estimation of regime, gaze pattern, and model parameters are implemented using a Markov chain Monte Carlo method. Experiments on four-person conversations confirm accurate gaze estimation and the effectiveness of the framework toward identification of the conversation structures.