A hybrid approach of NN and HMM for facial emotion classification

Authors:
Tianming Hu;Liyanage C. De Silva;Kuntal Sengupta
Affiliations:
Department of Electrical and Computer Engineering, National University of Singapore, 10 Kent Ridge Crescent, 119260 Singapore, Singapore;Department of Electrical and Computer Engineering, National University of Singapore, 10 Kent Ridge Crescent, 119260 Singapore, Singapore;Department of Electrical and Computer Engineering, National University of Singapore, 10 Kent Ridge Crescent, 119260 Singapore, Singapore
Venue:
Pattern Recognition Letters
Year:
2002

Citing 2
Cited 3

Classifying Facial Actions

IEEE Transactions on Pattern Analysis and Machine Intelligence
Comparison Between Geometry-Based and Gabor-Wavelets-Based Facial Expression Recognition Using Multi-Layer Perceptron

FG '98 Proceedings of the 3rd. International Conference on Face & Gesture Recognition

A similarity-based neural network for facial expression analysis

Pattern Recognition Letters
Zero knowledge hidden Markov model inference

Pattern Recognition Letters
Speech enhancement with inventory style speech resynthesis

IEEE Transactions on Audio, Speech, and Language Processing

Quantified Score

Hi-index	0.10

Visualization

Abstract

Neural networks (NNs) are often combined with Hidden Markov Models (HMMs) in speech recognition for achieving superior performance. In this paper, this hybrid approach is employed in facial emotion classification. Gabor wavelets are employed to extract features from difference images obtained by subtracting the first frame showing a frontal face from the current frame. The NN, which takes the form of Multilayer perceptron (MLP), is used to classify the feature vector into different states of a HMM of a certain emotion sequence, i.e., neutral, intermediate and peak. In addition to using 1-0 as targets for the NN, a heuristic strategy of assigning variable targets 1-x-0 has also been applied. After training, we interpret the output values of the NN as the posterior of the HMM state and directly apply the Viterbi algorithm to these values to estimate the best state path. The experiments show that with variable targets for the NN, the HMM gives better results than that with 1-0 targets. The best HMM results are obtained for x = 0.8 in 1-x-0.