Combining audio and video for detection of spontaneous emotions

  • Authors:

  • Rok Gajšek;Vitomir Štruc;Simon Dobrišek;Janez Žibert;France Mihelič;Nikola Pavešić

  • Affiliations:

  • Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia;Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia;Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia;Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia;Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia;Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia

  • Venue:
  • BioID_MultiComm'09 Proceedings of the 2009 joint COST 2101 and 2102 international conference on Biometric ID management and multimodal communication
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

The paper presents our initial attempts in building an audio video emotion recognition system. Both, audio and video sub-systems are discussed, and description of the database of spontaneous emotions is given. The task of labelling the recordings from the database according to different emotions is discussed and the measured agreement between multiple annotators is presented. Instead of focusing on the prosody in audio emotion recognition, we evaluate the possibility of using linear transformations (CMLLR) as features. The classification results from audio and video sub-systems are combined using sum rule fusion and the increase in recognition results, when using both modalities, is presented.