Audio-visual based emotion recognition-a new approach

Authors:
Mingli Song;Jiajun Bu;Chun Chen;Nan Li
Affiliations:
College of Computer Science, Zhejiang University, Hangzhou, P.R.China;College of Computer Science, Zhejiang University, Hangzhou, P.R.China;College of Computer Science, Zhejiang University, Hangzhou, P.R.China;College of Computer Science, Zhejiang University, Hangzhou, P.R.China
Venue:
CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
Year:
2004

Citing 5
Cited 21

Active Appearance Models

ECCV '98 Proceedings of the 5th European Conference on Computer Vision-Volume II - Volume II
Subtly Different Facial Expression Recognition and Expression Intensity Estimation

CVPR '98 Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Facial Expression Space Learning

PG '02 Proceedings of the 10th Pacific Conference on Computer Graphics and Applications
Facial expression recognition from video sequences: temporal and static modeling

Computer Vision and Image Understanding - Special issue on Face recognition
Learning bayesian network classifiers for facial expression recognition using both labeled and unlabeled data

CVPR'03 Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition

Affective multimodal human-computer interaction

Proceedings of the 13th annual ACM international conference on Multimedia
Multimodal analysis of recorded video for e-learning

Proceedings of the 13th annual ACM international conference on Multimedia
Audio-visual emotion recognition in adult attachment interview

Proceedings of the 8th international conference on Multimodal interfaces
Human computing and machine understanding of human behavior: a survey

Proceedings of the 8th international conference on Multimodal interfaces
Multimodal human-computer interaction: A survey

Computer Vision and Image Understanding
A survey of affect recognition methods: audio, visual and spontaneous expressions

Proceedings of the 9th international conference on Multimodal interfaces
Detecting communication errors from visual cues during the system's conversational turn

Proceedings of the 9th international conference on Multimodal interfaces
Psychological responses to simulated displays of mismatched emotional expressions

Interacting with Computers
A robust multimodal approach for emotion recognition

Neurocomputing
Supporting Ubiquitous Collaboration with Real-Time Facial Animation

Computer Supported Cooperative Work in Design IV
Acoustic feature selection for automatic emotion recognition from speech

Information Processing and Management: an International Journal
Static vs. dynamic modeling of human nonverbal behavior from multiple cues and modalities

Proceedings of the 2009 international conference on Multimodal interfaces
Applying Affect Recognition in Serious Games: The PlayMancer Project

MIG '09 Proceedings of the 2nd International Workshop on Motion in Games
Human computing and machine understanding of human behavior: a survey

ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing
Audio-visual spontaneous emotion recognition

ICMI'06/IJCAI'07 Proceedings of the ICMI 2006 and IJCAI 2007 international conference on Artifical intelligence for human computing
Theories, methods and current research on emotions in library and information science, information retrieval and human-computer interaction

Information Processing and Management: an International Journal
Emotion recognition using bimodal data fusion

Proceedings of the 12th International Conference on Computer Systems and Technologies
Sketch based facial expression recognition using graphics hardware

ACII'05 Proceedings of the First international conference on Affective Computing and Intelligent Interaction
Semi-videoconference system using real-time wireless technologies

ICESS'04 Proceedings of the First international conference on Embedded Software and Systems
A multimodal fuzzy inference system using a continuous facial expression representation for emotion detection

Proceedings of the 14th ACM international conference on Multimodal interaction
Emotion recognition using facial and audio features

Proceedings of the 15th ACM on International conference on multimodal interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

Emotion recognition is one of the latest challenges in intelligent human/computer communication. Most of previous work on emotion recognition focused on extracting emotions from visual or audio information separately. A novel approach is presented in this paper, including both visual and audio from video clips, to recognize the human emotion. The Facial Animation Parameters (FAPs) compliant facial feature tracking based on active appearance model is performed on the video to generate two vector stream which represent the expression feature and the visual speech one. Combined with the visual vectors, the audio vector is extracted in terms of low level features. Then, a tripled Hidden Markov Model is introduced to perform the recognition which allows the state asynchrony of the audio and visual observation sequences while preserving their natural correlation over time. The experimental results show that this approach outperforms only using visual or audio separately.