HandySinger: expressive singing voice morphing using personified hand-puppet interface

Authors:
Tomoko Yonezawa;Noriko Suzuki;Kenji Mase;Kiyoshi Kogure
Affiliations:
ATR IRC Labs, Hikari-dai, Seika-cho, Kyoto, Japan;ATR MIS Labs, Hikari-dai, Seika-cho, Kyoto, Japan;Nagoya University, Furoh, Chikusa, Nagoya, Japan;ATR IRC Labs, Hikari-dai, Seika-cho, Kyoto, Japan
Venue:
NIME '05 Proceedings of the 2005 conference on New interfaces for musical expression
Year:
2005

Citing 3
Cited 5

Design of virtual 3D instruments for musical interaction

Proceedings of the 1999 conference on Graphics interface '99
Context-aware sensor-doll as a music expression device

CHI '01 Extended Abstracts on Human Factors in Computing Systems
Glove-talk II - a neural-network interface which maps gestures to parallel formant speech synthesizer controls

IEEE Transactions on Neural Networks

Cross-modal coordination of expressive strength between voice and gesture for personified media

Proceedings of the 8th international conference on Multimodal interfaces
Speech-driven embodied entrainment character system with hand motion input in mobile environment

Proceedings of the 9th international conference on Multimodal interfaces
Perceptual continuity and naturalness of expressive strength in singing voices based on speech morphing

EURASIP Journal on Audio, Speech, and Music Processing
The "one-person choir": A multidisciplinary approach to the development of an embodied human-computer interface

Computer Music Journal
Tangible and body-related interaction techniques for a singing voice synthesis installation

Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

The HandySinger system is a personified tool developed to naturally express a singing voice controlled by the gestures of a hand puppet. Assuming that a singing voice is a kind of musical expression, natural expressions of the singing voice are important for personification. We adopt a singing voice morphing algorithm that effectively smoothes out the strength of expressions delivered with a singing voice. The system's hand puppet consists of a glove with seven bend sensors and two pressure sensors. It sensitively captures the user's motion as a personified puppet's gesture. To synthesize the different expressional strengths of a singing voice, the "normal" (without expression) voice of a particular singer is used as the base of morphing, and three different expressions, "dark," "whisper" and "wet," are used as the target. This configuration provides musically expressed controls that are intuitive to users. In the experiment, we evaluate whether 1) the morphing algorithm interpolates expressional strength in a perceptual sense, 2) the handpuppet interface provides gesture data at sufficient resolution, and 3) the gestural mapping of the current system works as planned.