A statistical approach to analyzing sound tracings

Authors:
Kristian Nymoen;Jim Torresen;Rolf Inge Godøy;Alexander Refsum Jensenius
Affiliations:
Department of Informatics, University of Oslo, Oslo, Norway;Department of Informatics, University of Oslo, Oslo, Norway;Department of Musicology, University of Oslo, Oslo, Norway;Department of Musicology, University of Oslo, Oslo, Norway
Venue:
CMMR'11 Proceedings of the 8th international conference on Speech, Sound and Music Processing: embracing research in India
Year:
2011

Citing 9
Cited 2

Mappings and metaphors in auditory displays: An experimental assessment

ACM Transactions on Applied Perception (TAP)
Gestural-Sonorous Objects: embodied extensions of Schaeffer's conceptual apparatus

Organised Sound
Embodied Music Cognition and Mediation Technology

Embodied Music Cognition and Mediation Technology
Semiotics of Sounds Evoking Motions: Categorization and Acoustic Features

Computer Music Modeling and Retrieval. Sense of Sounds
Instrumental listening: Sonic gesture as design principle

Organised Sound
Chunking Sound for Musical Analysis

Computer Music Modeling and Retrieval. Genesis of Meaning in Sound and Music
Analyzing sound tracings: a multimodal approach to music information retrieval

MIRUM '11 Proceedings of the 1st international ACM workshop on Music information retrieval with user-centered and multimodal strategies
Towards a gesture-sound cross-modal analysis

GW'09 Proceedings of the 8th international conference on Gesture in Embodied Communication and Human-Computer Interaction
Effects of spectral features of sound on gesture type and timing

GW'11 Proceedings of the 9th international conference on Gesture and Sign Language in Human-Computer Interaction and Embodied Communication

Effects of spectral features of sound on gesture type and timing

GW'11 Proceedings of the 9th international conference on Gesture and Sign Language in Human-Computer Interaction and Embodied Communication
Analyzing correspondence between sound objects and body motion

ACM Transactions on Applied Perception (TAP)

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents an experiment on sound tracing, meaning an experiment on how people relate motion to sound. 38 participants were presented with 18 short sounds, and instructed to move their hands in the air while acting as though the sound was created by their hand motion. The hand motion of the participants was recorded, and has been analyzed using statistical tests, comparing results between different sounds, between different subjects, and between different sound classes. We have identified several relationships between sound and motion which are present in the majority of the subjects. A clear distinction was found in onset acceleration for motion to sounds with an impulsive dynamic envelope compared to non-impulsive sounds. Furthermore, vertical movement has been shown to be related to sound frequency, both in terms of spectral centroid and pitch. Moreover, a significantly higher amount of overall acceleration was observed for non-pitched sounds as compared to pitched sounds.