American sign language recognition with the kinect
ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Proceedings of the 12th International Conference on Interaction Design and Children
Hidden Markov model for human to computer interaction: a study on human hand gesture recognition
Artificial Intelligence Review
| Hi-index | 0.00 |
We perform real-time American Sign Language (ASL) phrase verification for an educational game, CopyCat, which is designed to improve deaf children's signing skills. Taking advantage of context information in the game we verify a phrase, using Hidden Markov Models (HMMs), by applying a rejection threshold on the probability of the observed sequence for each sign in the phrase. We tested this approach using 1204 signed phrase samples from 11 deaf children playing the game during the phase two deployment of CopyCat. The CopyCat data set is particularly challenging because sign samples are collected during live game play and contain many variations in signing and disfluencies. We achieved a phrase verification accuracy of 83% compared to 90% real-time performance by a sign linguist. We report on the techniques required to reach this level of performance.