Automatic Speech-Lip Synchronization System for 3D Animation

  • Authors:
  • Juan Monroy;Francisco Bellas;Richard J. Duro;Ruben Lopez;Antonio Puentes;Jacques Isaac

  • Affiliations:
  • Integrated Group for Engineering Research, Universidade da Coruna, Ferrol, Spain 15403;Integrated Group for Engineering Research, Universidade da Coruna, Ferrol, Spain 15403;Integrated Group for Engineering Research, Universidade da Coruna, Ferrol, Spain 15403;Bren Entertainment S.A., Filmax Entertainment, Santiago de Compostela, Spain;Bren Entertainment S.A., Filmax Entertainment, Santiago de Compostela, Spain;Bren Entertainment S.A., Filmax Entertainment, Santiago de Compostela, Spain

  • Venue:
  • KES '09 Proceedings of the 13th International Conference on Knowledge-Based and Intelligent Information and Engineering Systems: Part I
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

In the 3D animation field, the quality of productions is continuously increasing. It is a very active market with a high level of competitiveness where modest companies, in terms of budget, must reach a balance between the resources they can apply and the economic investment in a given production. Consequently, the automation of manual design processes, which are normally highly time-consuming, has become a crucial research topic for 3D animation studios. The work we are describing here presents one of these automatic tools, specifically focused on the synchronization of the speech and the lip movement of the characters, a process that is called lipsync . We have developed a very robust and accurate speech recognition module that together with a knowledge-based system, autonomously provides lipsync results. Additionally, the system has been integrated in the production plan of a 3D animation company, leading to drastic operator time reductions.