Generating finely synchronized gesture and speech for humanoid robots: a closed-loop approach

  • Authors:

  • Maha Salem;Stefan Kopp;Frank Joublin

  • Affiliations:

  • Bielefeld University, Bielefeld, Germany;Bielefeld University, Bielefeld, Germany;Honda Research Institute Europe, Offenbach, Germany

  • Venue:
  • Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

Previous work focusing on the production of communicative robot gesture has not sufficiently addressed the challenge of speech-gesture synchrony. We propose a novel multimodal scheduler that comprises two features to improve the synchronization process. First, the scheduler integrates an experimentally fitted forward model at the behavior planning stage to provide a more accurate estimation of the robot's gesture preparation time. Second, the model incorporates a feedback-based adaptation mechanism which allows for an on-line adjustment of the synchronization of the two modalities during execution. In this way, it represents the first closed-loop approach to speech and gesture generation for humanoid robots.