Effect of presenting video as a baseline during an american sign language animation user study

  • Authors:

  • Pengfei Lu;Hernisa Kacorri

  • Affiliations:

  • The Graduate Center, CUNY, City University of New York, New York, NY, USA;The Graduate Center, CUNY, City University of New York, New York, NY, USA

  • Venue:
  • Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Animations of American Sign Language (ASL) have accessibility benefits for many signers with lower levels of written language literacy. Our lab has conducted several prior studies to evaluate synthesized ASL animations by asking native signers to watch different versions of animations and to answer comprehension and subjective questions about them. As an upper baseline, we used an animation of a virtual human carefully created by a human animator who is a native ASL signer. Considering whether to instead use videos of human signers as an upper baseline, we wanted to quantify how including a video upper baseline would affect how participants evaluate the ASL animations presented in a study. In this paper, we replicate a user study we conducted two years ago, with one difference: replacing our original animation upper baseline with a video of a human signer. We found that adding a human video upper baseline depressed the subjective Likert-scale scores that participants assign to the other stimuli (the synthesized animations) in the study when viewed side-by-side. This paper provides methodological guidance for how to design user studies evaluating sign language animations and facilitates comparison of studies that have used different upper baselines.