Analyzing sound tracings: a multimodal approach to music information retrieval

Authors:
Kristian Nymoen;Baptiste Caramiaux;Mariusz Kozak;Jim Torresen
Affiliations:
University of Oslo, Oslo, Norway;IRCAM, CNRS, Paris, France;University of Chicago, Chicago, IL, USA;University of Oslo, Oslo, Norway
Venue:
MIRUM '11 Proceedings of the 1st international ACM workshop on Music information retrieval with user-centered and multimodal strategies
Year:
2011

Citing 6
Cited 3

Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques

International Journal of Human-Computer Studies - Application of affective computing in human—Computer interaction
Gestural-Sonorous Objects: embodied extensions of Schaeffer's conceptual apparatus

Organised Sound
Semiotics of Sounds Evoking Motions: Categorization and Acoustic Features

Computer Music Modeling and Retrieval. Sense of Sounds
Instrumental listening: Sonic gesture as design principle

Organised Sound
Environmental sound perception: metadescription and modeling based on independent primary studies

EURASIP Journal on Audio, Speech, and Music Processing - Special issue on scalable audio-content analysis
Towards a gesture-sound cross-modal analysis

GW'09 Proceedings of the 8th international conference on Gesture in Embodied Communication and Human-Computer Interaction

1st international ACM workshop on music information retrieval with user-centered and multimodal strategies (MIRUM)

MM '11 Proceedings of the 19th ACM international conference on Multimedia
A statistical approach to analyzing sound tracings

CMMR'11 Proceedings of the 8th international conference on Speech, Sound and Music Processing: embracing research in India
Analyzing correspondence between sound objects and body motion

ACM Transactions on Applied Perception (TAP)

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper investigates differences in the gestures people relate to pitched and non-pitched sounds respectively. An experiment has been carried out where participants were asked to move a rod in the air, pretending that moving it would create the sound they heard. By applying and interpreting the results from Canonical Correlation Analysis we are able to determine both simple and more complex correspondences between features of motion and features of sound in our data set. Particularly, the presence of a distinct pitch seems to influence how people relate gesture to sound. This identification of salient relationships between sounds and gestures contributes as a multi-modal approach to music information retrieval.