Tracking eyebrows and head gestures associated with spoken prosody

  • Authors:

  • Jeesun Kim;Erin Cvejic;Chris Davis

  • Affiliations:

  • The MARCS Institute, University of Western Sydney, Australia;The MARCS Institute, University of Western Sydney, Australia and School of Psychiatry, University of New South Wales, Australia;The MARCS Institute, University of Western Sydney, Australia

  • Venue:
  • Speech Communication
  • Year:
  • 2014

Quantified Score

Hi-index 0.00

Visualization

Abstract

Although it is clear that eyebrow and head movements are in some way associated with spoken prosody, the precise form of this association is unclear. To examine this, eyebrow and head movements were recorded from six talkers producing 30 sentences (with two repetitions) in three prosodic conditions (Broad focus, Narrow focus and Echoic question) in a face to face dialogue exchange task. Movement displacement and peak velocity were measured for the prosodically marked constituents (critical region) as well as for the preceding and following regions. The amount of eyebrow movement in the Narrow focus and Echoic question conditions tended to be larger at the beginning of an utterance (in the pre-critical and critical regions) than at the end (in the post-critical region). Head rotation (nodding) tended to occur later, being maximal in the critical region and still occurring often in the post-critical one. For eyebrow movements, peak velocity tended to distinguish the regions better than the displacement measure. The extent to which eyebrow and head movements co-occurred was also examined. Compared to broad focussed condition, both movement types occurred more often in the narrow focussed and echoic question ones. When these double movements occurred in narrow focused utterances, brow raises tended to begin before the onset of the critical constituent and reach a peak displacement at the time of the critical constituent, whereas rigid pitch movements tended to begin at the time of critical constituent and reach peak displacement after this region. The pattern for echoic questions was similar for eyebrow motion however head rotations tended to begin earlier compared to the narrow focus condition. These results are discussed in terms of the differences these types of visual cues may have in production and perception.