Analyzing correspondence between sound objects and body motion

Authors:
Kristian Nymoen;Rolf Inge Godøy;Alexander Refsum Jensenius;Jim Torresen
Affiliations:
University of Oslo, Oslo, Norway;University of Oslo, Oslo, Norway;University of Oslo, Oslo, Norway;University of Oslo, Oslo, Norway
Venue:
ACM Transactions on Applied Perception (TAP)
Year:
2013

Citing 12
Cited 0

Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques

International Journal of Human-Computer Studies - Application of affective computing in human—Computer interaction
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
YALE: rapid prototyping for complex data mining tasks

Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
Gestural-Sonorous Objects: embodied extensions of Schaeffer's conceptual apparatus

Organised Sound
Embodied Music Cognition and Mediation Technology

Embodied Music Cognition and Mediation Technology
Semiotics of Sounds Evoking Motions: Categorization and Acoustic Features

Computer Music Modeling and Retrieval. Sense of Sounds
Instrumental listening: Sonic gesture as design principle

Organised Sound
Images of sonic objects

Organised Sound
Analyzing sound tracings: a multimodal approach to music information retrieval

MIRUM '11 Proceedings of the 1st international ACM workshop on Music information retrieval with user-centered and multimodal strategies
Towards a gesture-sound cross-modal analysis

GW'09 Proceedings of the 8th international conference on Gesture in Embodied Communication and Human-Computer Interaction
A statistical approach to analyzing sound tracings

CMMR'11 Proceedings of the 8th international conference on Speech, Sound and Music Processing: embracing research in India
Effects of spectral features of sound on gesture type and timing

GW'11 Proceedings of the 9th international conference on Gesture and Sign Language in Human-Computer Interaction and Embodied Communication

Quantified Score

Hi-index	0.00

Visualization

Abstract

Links between music and body motion can be studied through experiments called sound-tracing. One of the main challenges in such research is to develop robust analysis techniques that are able to deal with the multidimensional data that musical sound and body motion present. The article evaluates four different analysis methods applied to an experiment in which participants moved their hands following perceptual features of short sound objects. Motion capture data has been analyzed and correlated with a set of quantitative sound features using four different methods: (a) a pattern recognition classifier, (b) t-tests, (c) Spearman's ρ correlation, and (d) canonical correlation. This article shows how the analysis methods complement each other, and that applying several analysis techniques to the same data set can broaden the knowledge gained from the experiment.