Multimodal detection of salient behaviors of approach-avoidance in dyadic interactions

Authors:
Bo Xiao;Panayiotis Georgiou;Brian Baucom;Shrikanth Narayanan
Affiliations:
University of Southern California, Los Angeles, CA, USA;University of Southern California, Los Angeles, CA, USA;University of Southern California, Los Angeles, CA, USA;University of Southern California, Los Angeles, CA, USA
Venue:
Proceedings of the 14th ACM international conference on Multimodal interaction
Year:
2012

Citing 3
Cited 1

Two-frame motion estimation based on polynomial expansion

SCIA'03 Proceedings of the 13th Scandinavian conference on Image analysis
"That's aggravating, very aggravating": is it possible to classify behaviors in couple interactions using automatically derived lexical features?

ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part I
Articulated pose estimation with flexible mixtures-of-parts

CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition

Video analysis of approach-avoidance behaviors of teenagers speaking with virtual agents

Proceedings of the 15th ACM on International conference on multimodal interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

Approach-Avoidance (AA) coding is a measure of involvement and immediacy in human dyadic interactions. We focus on analyzing the salient events in interactions that trigger change points in AA code in time, as perceived by domain experts. We employ coarse level visual cues associated with body parts, as well as vocal energy features. Motion vector extraction and body pose estimation techniques are used for extracting visual cues. Functionals of these cues are used as features for SVM based machine learning experiments. We found that the coder's judgments on salient events are related to the short time interval preceding the labeling. We also show that visual cues are the main information source for decision making on salient AA events, and that considering the information from a subset of body parts provides the same information as considering the full set. The mean of absolute value and standard deviation of motion streams are the most effective functionals as feature. We achieve an F-score of 0.55 in detecting salient events using cross-validation with a one-subject-out approach.