Feature extraction from faces using deformable templates
International Journal of Computer Vision
Active shape models—their training and application
Computer Vision and Image Understanding
Neural network design
Video Rewrite: driving visual speech with audio
Proceedings of the 24th annual conference on Computer graphics and interactive techniques
Lip movement synthesis from speech based on hidden Markov models
Speech Communication - Special issue on auditory-visual speech processing
FERSA: Lip-Synchronous Animation
ICSC '95 Proceedings of the Third International Computer Science Conference on Image Analysis Applications and Computer Graphics
Sample-Based Synthesis of Photo-Realistic Talking Heads
CA '98 Proceedings of the Computer Animation
MikeTalk: A Talking Facial Display Based on Morphing Visemes
CA '98 Proceedings of the Computer Animation
A Speech Driven Talking Head System Based on a Single Face Image
PG '99 Proceedings of the 7th Pacific Conference on Computer Graphics and Applications
3D Modeling and Tracking of Human Lip Motions
ICCV '98 Proceedings of the Sixth International Conference on Computer Vision
IEEE Transactions on Circuits and Systems for Video Technology
Geometry-Driven Photorealistic Facial Expression Synthesis
IEEE Transactions on Visualization and Computer Graphics
| Hi-index | 0.10 |
Mouth images are difficult to synthesize because they vary greatly according to different illumination, size and shape of mouth opening, and especially visibility of teeth and tongue. Conventional approaches such as manipulating 3D model or warping images do not produce very realistic animation. To overcome these difficulties, we describe a method of producing large variations of mouth shape and gray-level appearance using a compact parametric appearance model, which represents both shape and gray-level appearance. We find the high correlation between shape model parameters and gray-level model parameters, and design a shape appearance dependence mapping (SADM) strategy that converts one to the other. Once mouth shape parameters are derived from speech analysis, a proper full mouth appearance can be reconstructed with SADM. Some synthetic results of representative mouth appearance are shown in our experiments, they are very close to real mouth images. The proposed technique can be integrated into a speech-driven face animation system. In effect, SADM can synthesize not only the mouth image but also different kinds of dynamic facial texture, such as furrow, dimple and cheekbone shadows.