Effects of perturbation and prosody on the coordination of speech and gesture

  • Authors:

  • Heather Leavy Rusiewicz;Susan Shaiman;Jana M. Iverson;Neil Szuminsky

  • Affiliations:

  • Duquesne University, Department of Speech-Language Pathology, 412 Fisher Hall, 600 Forbes Avenue, Pittsburgh, PA 15282, USA;University of Pittsburgh, Department of Communication Science and Disorders, 4033 Forbes Tower, 3600 Forbes at Atwood Street, Pittsburgh, PA 15260, USA;University of Pittsburgh, Department of Psychology, 3415 Sennott Square, 210 South Bouquet Street, Pittsburgh, PA 15260, USA;University of Pittsburgh, Department of Communication Science and Disorders, 4033 Forbes Tower, 3600 Forbes at Atwood Street, Pittsburgh, PA 15260, USA

  • Venue:
  • Speech Communication
  • Year:
  • 2014

Quantified Score

Hi-index 0.00

Visualization

Abstract

The temporal alignment of speech and gesture is widely acknowledged as primary evidence of the integration of spoken language and gesture systems. Yet there is a disconnect between the lack of experimental research on the variables that affect the temporal relationship of speech and gesture and the overwhelming acceptance that speech and gesture are temporally coordinated. Furthermore, the mechanism of the temporal coordination of speech and gesture is poorly represented. Recent experimental research suggests that gestures overlap prosodically prominent points in the speech stream, though the effects of other variables such as perturbation of speech are not yet studied in a controlled paradigm. The purpose of the present investigation was to further investigate the mechanism of this interaction according to a dynamic systems framework. Fifteen typical young adults completed a task that elicited the production of contrastive prosodic stress on different syllable positions with and without delayed auditory feedback while pointing to corresponding pictures. The coordination of deictic gestures and spoken language was examined as a function of perturbation, prosody, and position of the target syllable. Results indicated that the temporal parameters of gesture were affected by all three variables. The findings suggest that speech and gesture may be coordinated due to internal pulse-based temporal entrainment of the two motor systems.