Combined support vector machines and hidden Markov models for modeling facial action temporal dynamics

  • Authors:
  • Michel F. Valstar;Maja Pantic

  • Affiliations:
  • Department of Computing, Imperial College London, London, England;Department of Computing, Imperial College London, London, England

  • Venue:
  • HCI'07 Proceedings of the 2007 IEEE international conference on Human-computer interaction
  • Year:
  • 2007

Quantified Score

Hi-index 0.01

Visualization

Abstract

The analysis of facial expression temporal dynamics is of great importance for many real-world applications. Being able to automatically analyse facial muscle actions (Action Units, AUs) in terms of recognising their neutral, onset, apex and offset phases would greatly benefit application areas as diverse as medicine, gaming and security. The base system in this paper uses Support Vector Machines (SVMs) and a set of simple geometrical features derived from automatically detected and tracked facial feature point data to segment a facial action into its temporal phases. We propose here two methods to improve on this base system in terms of classification accuracy. The first technique describes the original time-independent set of features over a period of time using polynomial parametrisation. The second technique replaces the SVM with a hybrid SVM/Hidden Markov Model (HMM) classifier to model time in the classifier. Our results show that both techniques contribute to an improved classification accuracy. Modeling the temporal dynamics by the hybrid SVM-HMM classifier attained a statistically significant increase of recall and precision by 4.5% and 7.0%, respectively.