Manual annotation and automatic image processing of multimodal emotional behaviors: validating the annotation of TV interviews

Authors:
J. -C. Martin;G. Caridakis;L. Devillers;K. Karpouzis;S. Abrilian
Affiliations:
LIMSI-CNRS, Orsay Cedex, France 91403;Image, Video and Multimedia Systems Lab, National Technical, University of Athens, Athens, Greece 157 80;LIMSI-CNRS, Orsay Cedex, France 91403;Image, Video and Multimedia Systems Lab, National Technical, University of Athens, Athens, Greece 157 80;LIMSI-CNRS, Orsay Cedex, France 91403
Venue:
Personal and Ubiquitous Computing
Year:
2009

Citing 9
Cited 2

Emotional speech: towards a new generation of databases

Speech Communication - Special issue on speech and emotion
Experimental study of affect bursts

Speech Communication - Special issue on speech and emotion
Levels of representation in the annotation of emotion for the specification of expressivity in ECAs

Lecture Notes in Computer Science
Implementing expressive gesture synthesis for embodied conversational agents

GW'05 Proceedings of the 6th international conference on Gesture in Human-Computer Interaction and Simulation
Gesture-Based affective computing on motion capture data

ACII'05 Proceedings of the First international conference on Affective Computing and Intelligent Interaction
Towards unsupervised detection of affective body posture nuances

ACII'05 Proceedings of the First international conference on Affective Computing and Intelligent Interaction
Fusing face and body display for bi-modal emotion recognition: single frame analysis and multi-frame post integration

ACII'05 Proceedings of the First international conference on Affective Computing and Intelligent Interaction
Representing real-life emotions in audiovisual data with non basic emotional patterns and context features

ACII'05 Proceedings of the First international conference on Affective Computing and Intelligent Interaction
Annotating multimodal behaviors occurring during non basic emotions

ACII'05 Proceedings of the First international conference on Affective Computing and Intelligent Interaction

Estimation of behavioral user state based on eye gaze and head pose--application in an e-learning environment

Multimedia Tools and Applications
Probabilistic video-based gesture recognition using self-organizing feature maps

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

There have been a lot of psychological researches on emotion and nonverbal communication. Yet, these studies were based mostly on acted basic emotions. This paper explores how manual annotation and image processing can cooperate towards the representation of spontaneous emotional behavior in low-resolution videos from TV. We describe a corpus of TV interviews and the manual annotations that have been defined. We explain the image-processing algorithms that have been designed for the automatic estimation of movement quantity. Finally, we explore how image processing can be used for the validation of manual annotations.