Kernel fusion of audio and visual information for emotion recognition

Authors:
Yongjin Wang;Rui Zhang;Ling Guan;A. N. Venetsanopoulos
Affiliations:
Department of Electrical and Computer Engineering, Ryerson University, Toronto, Ontario, Canada;Department of Electrical and Computer Engineering, Ryerson University, Toronto, Ontario, Canada;Department of Electrical and Computer Engineering, Ryerson University, Toronto, Ontario, Canada;Department of Electrical and Computer Engineering, Ryerson University, Toronto, Ontario, Canada
Venue:
ICIAR'11 Proceedings of the 8th international conference on Image analysis and recognition - Volume Part II
Year:
2011

Citing 6
Cited 0

Nonlinear component analysis as a kernel eigenvalue problem

Neural Computation
Generalized Discriminant Analysis Using a Kernel Approach

Neural Computation
Audio-Visual Emotion Recognition Using Gaussian Mixture Models for Face and Voice

ISM '08 Proceedings of the 2008 Tenth IEEE International Symposium on Multimedia
Audio–Visual Affective Expression Recognition Through Multistream Fused HMM

IEEE Transactions on Multimedia
Recognizing Human Emotional State From Audiovisual Signals*

IEEE Transactions on Multimedia
An introduction to kernel-based learning algorithms

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

Effective analysis and recognition of human emotional behavior are important for achieving efficient and intelligent human computer interaction. This paper presents an approach for audiovisual based multimodal emotion recognition. The proposed solution integrates the audio and visual information by fusing the kernel matrices of respective channels through algebraic operations, followed by dimensionality reduction techniques to map the original disparate features to a nonlinearly transformed joint subspace. A hidden Markov model is employed for characterizing the statistical dependence across successive frames, and identifying the inherent temporal structure of the features. We examine the kernel fusion method at both feature and score levels. The effectiveness of the proposed method is demonstrated through extensive experimentation.