Audiovisual alignment in a face-to-face conversation translation framework

  • Authors:
  • Jerneja Žganec Gros;Aleš Mihelič

  • Affiliations:
  • Alpineon Research and Development, Ljubljana, Slovenia;Alpineon Research and Development, Ljubljana, Slovenia

  • Venue:
  • BioID_MultiComm'09 Proceedings of the 2009 joint COST 2101 and 2102 international conference on Biometric ID management and multimodal communication
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Recent improvements in audiovisual alignment for a translating videophone are presented. A method for audiovisual alignment in the target language is proposed and the process of audiovisual speech synthesis is described. The proposed method has been evaluated in the VideoTRAN translating videophone environment, where an H.323 software client translating videophone allows for the transmission and translation of a set of multimodal verbal and nonverbal clues in a multilingual face-to-face communication setting. An extension of subjective evaluation metrics of fluency and adequacy, which are commonly used in subjective machine translation evaluation tests, is proposed for usage in an audiovisual translation environment.