Tracking vertex flow and model adaptation for three-dimensional spatiotemporal face analysis

  • Authors:

  • Yi Sun;Xiaochen Chen;Matthew Rosato;Lijun Yin

  • Affiliations:

  • Department of Computer Science, The State University of New York at Binghamton, Binghamton, NY;UBS Investment Bank, Stamford, CT and Department of Computer Science, The State University of New York at Binghamton, Binghamton, NY;Department of Computer Science, The State University of New York at Binghamton, Binghamton, NY and IBM, Endicott, NY;Department of Computer Science, The State University of New York at Binghamton, Binghamton, NY

  • Venue:
  • IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans - Special issue on recent advances in biometrics
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Research in the areas of 3-D face recognition and 3-D facial expression analysis has intensified in recent years. However, most research has been focused on 3-D static data analysis. In this paper, we investigate the facial analysis problem using dynamic 3-D face model sequences. One of the major obstacles for analyzing such data is the lack of correspondences of features due to the variable number of vertices across individual models or 3-D model sequences. In this paper, we present an effective approach for establishing vertex correspondences using a tracking-model-based approach for vertex registration, coarse-to-fine model adaptation, and vertex motion trajectory (called vertex flow) estimation. We propose to establish correspondences across frame models based on a 2-D intermediary, which is generated using conformal mapping and a generic model adaptation algorithm. Based on our newly created 3-D dynamic face database, we also propose to use a spatiotemporal hidden Markov model (ST-HMM) that incorporates 3-D surface feature characterization to learn the spatial and temporal information of faces. The advantage of using 3-D dynamic data for face recognition has been evaluated by comparing our approach to three conventional approaches: 2-D-videobased temporal HMM model, conventional 2-D-texture-based approach (e.g., Gabor-wavelet-based approach), and static 3-D-model-based approaches. To further evaluate the usefulness of vertex flow and the adapted model, we have also applied a spatial-temporal face model descriptor for facial expression classification based on dynamic 3-D model sequences.