Naturalistic affective expression classification by a multi-stage approach based on hidden Markov models

Authors:
Hongying Meng;Nadia Bianchi-Berthouze
Affiliations:
UCL Interaction Centre, University College London, London, UK;UCL Interaction Centre, University College London, London, UK
Venue:
ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part II
Year:
2011

Citing 7
Cited 5

Affective computing

Affective computing
Automatic Analysis of Facial Expressions: The State of the Art

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions

IEEE Transactions on Pattern Analysis and Machine Intelligence
Survey on speech emotion recognition: Features, classification schemes, and databases

Pattern Recognition
Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space

IEEE Transactions on Affective Computing
The first audio/visual emotion challenge and workshop: an introduction

ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part II
Automatic Recognition of Non-Acted Affective Postures

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Robust continuous prediction of human emotions using multiscale dynamic cues

Proceedings of the 14th ACM international conference on Multimodal interaction
LSTM-Modeling of continuous emotions in an audiovisual affect recognition framework

Image and Vision Computing
Tracking continuous emotional trends of participants during affective dyadic interactions using body language and speech information

Image and Vision Computing
Audiovisual three-level fusion for continuous estimation of Russell's emotion circumplex

Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge
Shape-based modeling of the fundamental frequency contour for emotion detection in speech

Computer Speech and Language

Quantified Score

Hi-index	0.00

Visualization

Abstract

In naturalistic behaviour, the affective states of a person change at a rate much slower than the typical rate at which video or audio is recorded (e.g. 25fps for video). Hence, there is a high probability that consecutive recorded instants of expressions represent a same affective content. In this paper, a multi-stage automatic affective expression recognition system is proposed which uses Hidden Markov Models (HMMs) to take into account this temporal relationship and finalize the classification process. The hidden states of the HMMs are associated with the levels of affective dimensions to convert the classification problem into a best path finding problem in HMM. The system was tested on the audio data of the Audio/Visual Emotion Challenge (AVEC) datasets showing performance significantly above that of a one-stage classification system that does not take into account the temporal relationship, as well as above the baseline set provided by this Challenge. Due to the generality of the approach, this system could be applied to other types of affective modalities.