Recognizing combinations of facial action units with different intensity using a mixture of hidden markov models and neural network

Authors:
Mahmoud Khademi;Mohammad Taghi Manzuri-Shalmani;Mohammad Hadi Kiapour;Ali Akbar Kiaei
Affiliations:
DSP Lab, Sharif University of Technology, Tehran, Iran;DSP Lab, Sharif University of Technology, Tehran, Iran;DSP Lab, Sharif University of Technology, Tehran, Iran;DSP Lab, Sharif University of Technology, Tehran, Iran
Venue:
MCS'10 Proceedings of the 9th international conference on Multiple Classifier Systems
Year:
2010

Citing 11
Cited 0

Face Recognition by Elastic Bunch Graph Matching

IEEE Transactions on Pattern Analysis and Machine Intelligence
Automatic Analysis of Facial Expressions: The State of the Art

IEEE Transactions on Pattern Analysis and Machine Intelligence
Recognizing Action Units for Facial Expression Analysis

IEEE Transactions on Pattern Analysis and Machine Intelligence
PCA versus LDA

IEEE Transactions on Pattern Analysis and Machine Intelligence
Coding Facial Expressions with Gabor Wavelets

FG '98 Proceedings of the 3rd. International Conference on Face & Gesture Recognition
Comprehensive Database for Facial Expression Analysis

FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
Evaluation of Gabor-Wavelet-Based Facial Action Unit Recognition in Image Sequences of Increasing Complexity

FGR '02 Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition
Special issue on face recognition

Computer Vision and Image Understanding - Special issue on Face recognition
Two-Dimensional PCA: A New Approach to Appearance-Based Face Representation and Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines

IEEE Transactions on Image Processing
Human expression recognition from motion using a radial basis function network architecture

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

Facial Action Coding System consists of 44 action units (AUs) and more than 7000 combinations. Hidden Markov models (HMMs) classifier has been used successfully to recognize facial action units (AUs) and expressions due to its ability to deal with AU dynamics. However, a separate HMM is necessary for each single AU and each AU combination. Since combinations of AU numbering in thousands, a more efficient method will be needed. In this paper an accurate real-time sequence-based system for representation and recognition of facial AUs is presented. Our system has the following characteristics: 1) employing a mixture of HMMs and neural network, we develop a novel accurate classifier, which can deal with AU dynamics, recognize subtle changes, and it is also robust to intensity variations, 2) although we use an HMM for each single AU only, by employing a neural network we can recognize each single and combination AU, and 3) using both geometric and appearance-based features, and applying efficient dimension reduction techniques, our system is robust to illumination changes and it can represent the temporal information involved in formation of the facial expressions. Extensive experiments on Cohn-Kanade database show the superiority of the proposed method, in comparison with other classifiers.