Sample-Based Synthesis of Talking Heads

  • Authors:
  • Hans Peter Graf;Eric Cosatto

  • Affiliations:
  • -;-

  • Venue:
  • RATFG-RTS '01 Proceedings of the IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems (RATFG-RTS'01)
  • Year:
  • 2001

Quantified Score

Hi-index 0.00

Visualization

Abstract

Abstract: Synthesizing photo-realistic talking heads is a challenging problem, and so far all attempts using conventional computer graphics produced heads with a distinctly synthetic look. In order to look credible, a head must show a picture-perfect appearance, natural head movements, and good lip-sound synchronization. We use sample-based graphics to achieve more photo-realistic appearances than what is possible with the traditional approach of 3D modeling and texture mapping. For sample-based graphics, first parts of faces are cut from recorded images and are stored in a database. New sequences are then synthesized by integrating such parts into whole faces. With sufficient recorded data this approach produces by far the most naturally looking speech articulation. We are now integrating 3D modeling with the sample-based technique in order to enhance its flexibility. This allows, for example, showing the head over a much wider range of orientations.