Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces
IEEE Transactions on Pattern Analysis and Machine Intelligence
Performance-driven facial animation
SIGGRAPH '90 Proceedings of the 17th annual conference on Computer graphics and interactive techniques
A parallel relaxation method for quadratic programming problems with interval constraints
Proceedings of the international meeting on Linear/nonlinear iterative methods and verification of solution
Video Rewrite: driving visual speech with audio
Proceedings of the 24th annual conference on Computer graphics and interactive techniques
An eigenspace update algorithm for image analysis
Graphical Models and Image Processing
Proceedings of the 25th annual conference on Computer graphics and interactive techniques
EM algorithms for PCA and SPCA
NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
A morphable model for the synthesis of 3D faces
Proceedings of the 26th annual conference on Computer graphics and interactive techniques
Optical Flow Constraints on Deformable Models with Applications to Face Tracking
International Journal of Computer Vision
Real-time 3D model acquisition
Proceedings of the 29th annual conference on Computer graphics and interactive techniques
3-D Motion Estimation in Model-Based Facial Image Coding
IEEE Transactions on Pattern Analysis and Machine Intelligence
ECCV '98 Proceedings of the 5th European Conference on Computer Vision-Volume II - Volume II
Modeling, Tracking and Interactive Animation of Faces and Heads Using Input from Video
CA '96 Proceedings of the Computer Animation
ICCV '95 Proceedings of the Fifth International Conference on Computer Vision
Vision-based control of 3D facial animation
Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation
An Introduction to the Kalman Filter
An Introduction to the Kalman Filter
Weighted and Robust Incremental Method for Subspace Learning
ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
Deformation transfer for triangle meshes
ACM SIGGRAPH 2004 Papers
Spacetime faces: high resolution capture for modeling and animation
ACM SIGGRAPH 2004 Papers
High-Resolution, Real-time 3D Shape Acquisition
CVPRW '04 Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 3 - Volume 03
Face transfer with multilinear models
ACM SIGGRAPH 2005 Papers
Online Selection of Discriminative Tracking Features
IEEE Transactions on Pattern Analysis and Machine Intelligence
Universal capture - image-based facial animation for "The Matrix Reloaded"
SIGGRAPH '05 ACM SIGGRAPH 2005 Courses
On Linear Variational Surface Deformation Methods
IEEE Transactions on Visualization and Computer Graphics
Semi-supervised On-Line Boosting for Robust Tracking
ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
Face/Off: live facial puppetry
Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation
Robust single-view geometry and motion reconstruction
ACM SIGGRAPH Asia 2009 papers
Pose-space animation and transfer of facial details
Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation
The Digital Emily project: photoreal facial modeling and animation
ACM SIGGRAPH 2009 Courses
ACM SIGGRAPH 2010 papers
High resolution passive facial performance capture
ACM SIGGRAPH 2010 papers
Deformable Model Fitting by Regularized Landmark Mean-Shift
International Journal of Computer Vision
High-quality passive facial performance capture using anchor frames
ACM SIGGRAPH 2011 papers
Realtime performance-based facial animation
ACM SIGGRAPH 2011 papers
Lightweight binocular facial performance capture under uncontrolled lighting
ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH Asia 2012
High fidelity facial animation capture and retargeting with contours
Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation
Reconstructing detailed dynamic face geometry from monocular video
ACM Transactions on Graphics (TOG)
Special Section on CAD/Graphics 2013: SCAPE-based human performance reconstruction
Computers and Graphics
Hi-index | 0.00 |
We introduce a real-time and calibration-free facial performance capture framework based on a sensor with video and depth input. In this framework, we develop an adaptive PCA model using shape correctives that adjust on-the-fly to the actor's expressions through incremental PCA-based learning. Since the fitting of the adaptive model progressively improves during the performance, we do not require an extra capture or training session to build this model. As a result, the system is highly deployable and easy to use: it can faithfully track any individual, starting from just a single face scan of the subject in a neutral pose. Like many real-time methods, we use a linear subspace to cope with incomplete input data and fast motion. To boost the training of our tracking model with reliable samples, we use a well-trained 2D facial feature tracker on the input video and an efficient mesh deformation algorithm to snap the result of the previous step to high frequency details in visible depth map regions. We show that the combination of dense depth maps and texture features around eyes and lips is essential in capturing natural dialogues and nuanced actor-specific emotions. We demonstrate that using an adaptive PCA model not only improves the fitting accuracy for tracking but also increases the expressiveness of the retargeted character.