Mimicking video: real-time morphable 3D model fitting

Authors:
Kazuhiro Hiwada;Atsuto Maki;Akiko Nakashima
Affiliations:
Toshiba Corporation;Kyoto University;Toshiba Corporation
Venue:
Proceedings of the ACM symposium on Virtual reality software and technology
Year:
2003

Citing 10
Cited 1

Shape and motion from image streams under orthography: a factorization method

International Journal of Computer Vision
Synthesizing realistic facial expressions from photographs

Proceedings of the 25th annual conference on Computer graphics and interactive techniques
Quantitative association of vocal-tract and facial behavior

Speech Communication - Special issue on auditory-visual speech processing
A morphable model for the synthesis of 3D faces

Proceedings of the 26th annual conference on Computer graphics and interactive techniques
Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models

IEEE Transactions on Pattern Analysis and Machine Intelligence
Expression cloning

Proceedings of the 28th annual conference on Computer graphics and interactive techniques
Hyperpatches for 3D Model Acquisition and Tracking

IEEE Transactions on Pattern Analysis and Machine Intelligence
Trainable videorealistic speech animation

Proceedings of the 29th annual conference on Computer graphics and interactive techniques
Face Identification by Fitting a 3D Morphable Model Using Linear Shape and Texture Error Functions

ECCV '02 Proceedings of the 7th European Conference on Computer Vision-Part IV
Motion Regularization for Model-Based Head Tracking

ICPR '96 Proceedings of the International Conference on Pattern Recognition (ICPR '96) Volume III-Volume 7276 - Volume 7276

Face/Off: live facial puppetry

Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a new automatic scheme for tracking a 3D non-rigid object surface such as a human face in a real-time video sequence. We introduce a coordinate-oriented error minimization method for estimating the tracking parameters, whereas we base our algorithm on a morphable 3D model consisting of a combination of 3D linear bases, and show that it is extremely well suited to the task of fitting the 3D model to the target object in real time. The algorithm is straightforward, allowing the parameters of an object's pose and non-rigid motion to be computed in an integrated manner. Also, it is found that the illumination variability on the object surface, for instance due to the target motion, can be handled easily. Through the experiments we not only show that on-line tracking is indeed possible, but also demonstrate the effect of our technique of video mimicking.