3D deformable face tracking with a commodity depth camera

  • Authors:

  • Qin Cai;David Gallup;Cha Zhang;Zhengyou Zhang

  • Affiliations:

  • Communication and Collaboration Systems Group, Microsoft Research, Redmond, WA;Dept. of Computer Science, UNC at Chapel Hill, Chapel Hill, NC;Communication and Collaboration Systems Group, Microsoft Research, Redmond, WA;Communication and Collaboration Systems Group, Microsoft Research, Redmond, WA

  • Venue:
  • ECCV'10 Proceedings of the 11th European conference on computer vision conference on Computer vision: Part III
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Recently, there has been an increasing number of depth cameras available at commodity prices. These cameras can usually capture both color and depth images in real-time, with limited resolution and accuracy. In this paper, we study the problem of 3D deformable face tracking with such commodity depth cameras. A regularized maximum likelihood deformable model fitting (DMF) algorithm is developed, with special emphasis on handling the noisy input depth data. In particular, we present a maximum likelihood solution that can accommodate sensor noise represented by an arbitrary covariance matrix, which allows more elaborate modeling of the sensor's accuracy. Furthermore, an l1 regularization scheme is proposed based on the semantics of the deformable face model, which is shown to be very effective in improving the tracking results. To track facial movement in subsequent frames, feature points in the texture images are matched across frames and integrated into the DMF framework seamlessly. The effectiveness of the proposed method is demonstrated with multiple sequences with ground truth information.