A face robot for autonomous simplified musical notation reading and singing

  • Authors:

  • Chyi-Yeu Lin;Li-Chieh Cheng;Chang-Kuo Tseng;Hung-Yan Gu;Kuo-Liang Chung;Chin-Shyurng Fahn;Kai-Jay Lu;Chih-Cheng Chang

  • Affiliations:

  • Department of Mechanical Engineering, National Taiwan University of Science and Technology, 10607, Taipei, Taiwan;Department of Mechanical Engineering, National Taiwan University of Science and Technology, 10607, Taipei, Taiwan;Department of Mechanical Engineering, National Taiwan University of Science and Technology, 10607, Taipei, Taiwan;Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, 10607, Taipei, Taiwan;Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, 10607, Taipei, Taiwan;Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, 10607, Taipei, Taiwan;Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, 10607, Taipei, Taiwan;Institute of Automation and Control, National Taiwan University of Science and Technology, 10607, Taipei, Taiwan

  • Venue:
  • Robotics and Autonomous Systems
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

This research is aimed to devise an anthropomorphic robotic head with a human-like face and a sheet of artificial skin that can read a randomly composed simplified musical notation and sing the corresponding content of the song once. The face robot is composed of an artificial facial skin that can express a number of facial expressions via motions driven by internal servo motors. Two cameras, each of them installed inside each eyeball of the face, provide vision capability for reading simplified musical notations. Computer vision techniques are subsequently used to interpret simplified musical notations and lyrics of their corresponding songs. Voice synthesis techniques are implemented to enable the face robot to sing songs by enunciating synthesized sounds. Mouth patterns of the face robot will be automatically changed to match the emotions corresponding to the lyrics of the songs. The experiments show that the face robot can successfully read and then accurately sing a song which is assigned discriminately.