Real-time sensory substitution to enable players who are blind to play video games using whole body gestures

  • Authors:
  • Tony Morelli;Eelke Folmer

  • Affiliations:
  • University of Nevada, Reno;University of Nevada, Reno

  • Venue:
  • Proceedings of the 6th International Conference on Foundations of Digital Games
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Gesture-based interaction adds a new level of immersion to video games, but players who are blind are unable to play them as games use visual cues to indicate what gesture to provide and when. Though visual cues can be substituted with audio or haptic cues, this often requires access to the source code, which is not attainable for commercial games. We present a solution that uses real-time video analysis to detect the presence of a particular visual cue, which is then substituted with a vibrotactile cue that is provided with an external controller. A user study with 28 sighted participants with a popular commercial gesture based game showed no significant difference between visual and vibro-tactile cues, which demonstrates the feasibility of real-time sensory substitution as a cost-effective approach for making gesture-based video games accessible to players who are blind.