Shape Registration in Implicit Spaces Using Information Theory and Free Form Deformations
IEEE Transactions on Pattern Analysis and Machine Intelligence
High Resolution Tracking of Non-Rigid Motion of Densely Sampled 3D Data Using Harmonic Maps
International Journal of Computer Vision
Proceedings of the 31st DAGM Symposium on Pattern Recognition
Real-time 2D+3D facial action and expression recognition
Pattern Recognition
Realistic facial modeling and animation based on high resolution capture
ACIVS'07 Proceedings of the 9th international conference on Advanced concepts for intelligent vision systems
| Hi-index | 0.01 |
We present a novel hierarchical framework for high resolution, nonrigid facial expression tracking. The high quality dense point clouds of facial geometry moving at video speeds are acquired using a phase-shifting based structured light ranging technique. To use such data for temporal study of the subtle dynamics in expressions and for face recognition, an efficient nonrigid facial tracking algorithm is needed to establish intra-frame correspondences. In this paper, we propose such an algorithmic framework that uses a multi-resolution 3D deformable face model, and a hierarchical tracking scheme. This framework can not only track global facial motion that is caused by muscle action, but fit to subtler expression details that are generated by highly local skin deformations. Tracking of global deformations is performed efficiently on the coarse level of our face model with one thousand nodes, to recover the changes in a few intuitive parameters that control the motion of several deformable regions. In order to capture the complementary highly local deformations, we use a variational algorithm for non-rigid shape registration based on the integration of an implicit shape representation and the Free Form Deformations (FFD). Due to the strong implicit and explicit smoothness constraints imposed by the algorithm, the resulting registration/deformation field is smooth, continuous and gives dense one-to-one intra-frame correspondences. User-input sparse facial feature correspondences can also be incorporated as hard constraints in the optimization process, in order to guarantee high accuracy of the established correspondences. Extensive tracking experiments using the dynamic facial scan of five different subjects demonstrate the accuracy and efficiency of our proposed framework.