Developing a phoneme-based talking joystick for nonspeaking individuals

  • Authors:
  • Ha Trinh

  • Affiliations:
  • University of Dundee, Dundee, Scotland, United Kingdom

  • Venue:
  • ACM SIGACCESS Accessibility and Computing
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

This research investigates the potential of developing a novel phoneme-based assistive communication system for pre-literate individuals with severe speech and physical impairments. Using a force-feedback joystick-like game controller as the access tool, the system enables users to select forty-two phonemes (i.e., English sounds) used in literacy teaching and combine them together to generate spoken messages. What distinguishes this phoneme-based device from other communication systems currently available on the market is that it allows users who have not mastered literacy skills to create novel words and sentences without the need for a visual interface. Natural Language Processing (NLP) technologies, including phoneme-to-speech synthesis, phoneme-based disambiguation and prediction, and haptic force feedback technology are being incorporated into the device to improve its accessibility and usability.