A semi-automated system for accurate gaze coding in natural dyadic interactions

Authors:
Kenneth A. Funes Mora;Laurent Nguyen;Daniel Gatica-Perez;Jean-Marc Odobez
Affiliations:
Idiap Research Institute and École Polytechnique Fédérale de Lausanne, Martigny, Valais, Switzerland;Idiap Research Institute and École Polytechnique Fédérale de Lausanne, Martigny, Valais, Switzerland;Idiap Research Institute and École Polytechnique Fédérale de Lausanne, Martigny, Valais, Switzerland;Idiap Research Institute and École Polytechnique Fédérale de Lausanne, Martigny, Valais, Switzerland
Venue:
Proceedings of the 15th ACM on International conference on multimodal interaction
Year:
2013

Citing 8
Cited 1

Investigating automatic dominance estimation in groups from visual attention and speaking activity

ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
A 3D Face Model for Pose and Illumination Invariant Face Recognition

AVSS '09 Proceedings of the 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance
In the Eye of the Beholder: A Survey of Models for Eyes and Gaze

IEEE Transactions on Pattern Analysis and Machine Intelligence
Employing social gaze and speaking activity for automatic determination of the Extraversion trait

International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction
Conversation scene analysis based on dynamic Bayesian network and image-based gaze detection

International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction
Inferring human gaze from appearance via adaptive linear regression

ICCV '11 Proceedings of the 2011 International Conference on Computer Vision
Joint Depth and Color Camera Calibration with Distortion Correction

IEEE Transactions on Pattern Analysis and Machine Intelligence
Linking speaking and looking behavior patterns with group composition, perception, and performance

Proceedings of the 14th ACM international conference on Multimodal interaction

3D head pose and gaze tracking and their application to diverse multimodal tasks

Proceedings of the 15th ACM on International conference on multimodal interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we propose a system capable of accurately coding gazing events in natural dyadic interactions. Contrary to previous works, our approach exploits the actual continuous gaze direction of a participant by leveraging on remote RGB-D sensors and a head pose-independent gaze estimation method. Our contributions are: i) we propose a system setup built from low-cost sensors and a technique to easily calibrate these sensors in a room with minimal assumptions; ii) we propose a method which, provided short manual annotations, can automatically detect gazing events in the rest of the sequence; iii) we demonstrate on substantially long, natural dyadic data that high accuracy can be obtained, showing the potential of our system. Our approach is non-invasive and does not require collaboration from the interactors. These characteristics are highly valuable in psychology and sociology research.