Incremental polyphonic audio to score alignment using beat tracking for singer robots

Authors:
Takuma Otsuka;Kazumasa Murata;Kazuhiro Nakadai;Toru Takahashi;Kazunori Komatani;Tetsuya Ogata;Hiroshi G. Okuno
Affiliations:
Graduate School of Informatics, Kyoto University, Kyoto, Japan;Graduate School of Information Science and Engineering, Tokyo Institute of Technology, Tokyo, Japan;Honda Research Institute Japan, Co., Ltd., Wako, Saitama, Japan and Graduate School of Information Science and Engineering, Tokyo Institute of Technology;Graduate School of Informatics, Kyoto University, Kyoto, Japan;Graduate School of Informatics, Kyoto University, Kyoto, Japan;Graduate School of Informatics, Kyoto University, Kyoto, Japan;Graduate School of Informatics, Kyoto University, Kyoto, Japan
Venue:
IROS'09 Proceedings of the 2009 IEEE/RSJ international conference on Intelligent robots and systems
Year:
2009

Citing 3
Cited 2

Score following: state of the art and new developments

NIME '03 Proceedings of the 2003 conference on New interfaces for musical expression
A dancing robot for rhythmic social interaction

Proceedings of the ACM/IEEE international conference on Human-robot interaction
Toward Robotic Musicianship

Computer Music Journal

Real-time audio-to-score alignment using particle filter for coplayer music robots

EURASIP Journal on Advances in Signal Processing - Special issue on musical applications of real-time signal processing
A face robot for autonomous simplified musical notation reading and singing

Robotics and Autonomous Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

We aim at developing a singer robot capable of listening to music with its own ?ears? and interacting with a human's musical performance. Such a singer robot requires at least three functions: listening to the music, understanding what position in the music is being performed, and generating a singing voice. In this paper, we focus on the second function, that is, the capability to align an audio signal to its musical score represented symbolically. Issues underlying the score alignment problem are: (1) diversity in the sounds of various musical instruments, (2) difference between the audio signal and the musical score, (3) fluctuation in tempo of the musical performance. Our solutions to these issues are as follows: (1) the design of features based on a chroma vector in the 12-tone model and onset of the sound, (2) defining the rareness for each tone based on the idea that scarcely used tone is salient in the audio signal, and (3) the use of a switching Kalman filter for robust tempo estimation. The experimental result shows that our score alignment method improves the average of cumulative absolute errors in score alignment by 29% using 100 popular music tunes compared to the beat tracking without score alignment.