Gesture-driven American sign language phraselator

  • Authors:
  • Jose L. Hernandez-Rebollar

  • Affiliations:
  • George Washington University

  • Venue:
  • ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper describes a portable American Sign Language (ASL)-to-English phraselator. This wearable device is based on an Acceleglove originally developed for recognizing the hand alphabet, and a two-link arm skeleton that detects hand location and movement with respect to the body. Therefore, this phraselator is able to recognize finger-spelled words as well as hand gestures and translate them into spoken voice through a speech synthesizer. To speed-up the recognition process, a simple prediction algorithm has been introduced so the phraselator predicts words based on the current letter being inputted, or complete sentences based on the current sign being translated. The user selects the rest of the sentence (or word) by means of a predefined hand gesture for the phraselator to speak out the sentence in English or Spanish. New words of phrases are automatically added to the lexicon for future predictions.