Intelligible encoding of ASL image sequences at extremely low information rates
Papers from the second workshop Vol. 13 on Human and Machine Vision II
Conference Companion on Human Factors in Computing Systems
Sequential Operations in Digital Picture Processing
Journal of the ACM (JACM)
Text input for mobile devices: comparing model prediction to actual performance
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Computer
Power Evaluation of a Handheld Computer
IEEE Micro
Skin-Color Modeling and Adaptation
ACCV '98 Proceedings of the Third Asian Conference on Computer Vision-Volume II
An empirical study of typing rates on mini-QWERTY keyboards
CHI '05 Extended Abstracts on Human Factors in Computing Systems
Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning
IEEE Transactions on Pattern Analysis and Machine Intelligence
Robust region of interest coding for improved sign language telecommunication
IEEE Transactions on Information Technology in Biomedicine
Piecewise parametric interpolation for temporal compression of multijoint movement trajectories
IEEE Transactions on Information Technology in Biomedicine
Overview of the H.264/AVC video coding standard
IEEE Transactions on Circuits and Systems for Video Technology
Segmentation of the face and hands in sign language video sequences using color and motion cues
IEEE Transactions on Circuits and Systems for Video Technology
A web-based user survey for evaluating power saving strategies for deaf users of mobileASL
Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility
Hi-index | 0.00 |
We describe our system called MobileASL for real-time video communication on the current U.S. mobile phone network. The goal of MobileASL is to enable Deaf people to communicate with Sign Language over mobile phones by compressing and transmitting sign language video in real-time on an off-the-shelf mobile phone, which has a weak processor, uses limited bandwidth, and has little battery capacity. We develop several H.264-compliant algorithms to save system resources while maintaining ASL intelligibility by focusing on the important segments of the video. We employ a dynamic skin-based region-of-interest (ROI) that encodes the skin at higher quality at the expense of the rest of the video. We also automatically recognize periods of signing versus not signing and raise and lower the frame rate accordingly, a technique we call variable frame rate (VFR). We show that our variable frame rate technique results in a 47% gain in battery life on the phone, corresponding to an extra 68 minutes of talk time. We also evaluate our system in a user study. Participants fluent in ASL engage in unconstrained conversations over mobile phones in a laboratory setting. We find that the ROI increases intelligibility and decreases guessing. VFR increases the need for signs to be repeated and the number of conversational breakdowns, but does not affect the users' perception of adopting the technology. These results show that our sign language sensitive algorithms can save considerable resources without sacrificing intelligibility.