A new instrumented approach for translating the american sign language into sound and text

• Authors: Jose-Luis Hernandez-Rebollar;Nicholas Kyriakopoulos
• Venue: A new instrumented approach for translating the american sign language into sound and text
• Year: 2003

This dissertation discusses a novel approach for identifying isolated gestures of the American Sign Language and translating them into spoken words and text. The focus of the work is on sensing, gesture modeling, classification algorithm, system portability and lexicon scalability. The instrumented part of the system combines a novel device called The ‘AcceleGlove’ and a two-link arm skeleton. Sensing is done with accelerometers placed on the fingers, hand, and arm augmented with potentiometers on elbow and shoulder. The combination of signals from the glove and arm skeleton allows the detection of hand postures, hand location and hand movement without external (IR or RF) trackers. Gestures of the American Sign Language are broken down into unique sequences of phonemes called Poses and Movements. A pose refers to a static configuration of hand shape, palm orientation and hand location. These three simultaneous components are extracted from sensors and classified by three independent modules using linear classification. Movement refers to the trajectory of the hand when it travels from the initial to the final Pose, and is described by its curviness and direction. Modules were trained and tested independently on volunteers with different hand sizes and signing skills, with results up to 100% recognition rate on hand shape, orientation, and location. The system was tested with a subset of the American Sign Language dictionary comprised by 176 one-handed signs, classified using a search algorithm. The system is scalable, because it allows the addition of new signs without retraining, an improvement over classification based on Hidden Markov Models (HMM). The principles involved in detecting and classifying one-hand gestures are applicable to two-hand gestures. Although the gestures used for describing and evaluating the gesture recognition system are elements of the American Sign Language lexicon, the underlying principles are applicable to any set of gestures of similar phonetic structure.