Multi-stream confidence analysis for audio-visual affect recognition

Authors:
Zhihong Zeng;Jilin Tu;Ming Liu;Thomas S. Huang
Affiliations:
Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL;Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL;Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL;Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL
Venue:
ACII'05 Proceedings of the First international conference on Affective Computing and Intelligent Interaction
Year:
2005

Citing 8
Cited 3

Affective computing

Affective computing
Statistical Pattern Recognition: A Review

IEEE Transactions on Pattern Analysis and Machine Intelligence
Multimodal Human Emotion/Expression Recognition

FG '98 Proceedings of the 3rd. International Conference on Face & Gesture Recognition
Bimodal Emotion Recognition

FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
Joint processing of audio-visual information for the recognition of emotional expressions in human-computer interaction

Joint processing of audio-visual information for the recognition of emotional expressions in human-computer interaction
Bimodal HCI-related affect recognition

Proceedings of the 6th international conference on Multimodal interfaces
Audio-Visual Affect Recognition through Multi-Stream Fused HMM for HCI

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Face localization via hierarchical CONDENSATION with fisher boosting feature selection

CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition

Modeling naturalistic affective states via facial and vocal expressions recognition

Proceedings of the 8th international conference on Multimodal interfaces
Audio-visual spontaneous emotion recognition

ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing
Modeling naturalistic affective states via facial, vocal, and bodily expressions recognition

ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Changes in a speaker’s emotion are a fundamental component in human communication. Some emotions motivate human actions while others add deeper meaning and richness to human interactions. In this paper, we explore the development of a computing algorithm that uses audio and visual sensors to recognize a speaker’s affective state. Within the framework of Multi-stream Hidden Markov Model (MHMM), we analyze audio and visual observations to detect 11 cognitive/emotive states. We investigate the use of individual modality confidence measures as a means of estimating weights when combining likelihoods in the audio-visual decision fusion. Person-independent experimental results from 20 subjects in 660 sequences suggest that the use of stream exponents estimated on training data results in classification accuracy improvement of audio-visual affect recognition.