Automatic recognition of sign language subwords based on portable accelerometer and EMG sensors

Authors:
Yun Li;Xiang Chen;Jianxun Tian;Xu Zhang;Kongqiao Wang;Jihai Yang
Affiliations:
University of Science and Technology of China, Hefei, China;University of Science and Technology of China, Hefei, China;University of Science and Technology of China, Hefei, China;University of Science and Technology of China, Hefei, China;Nokia Research Center, NOKIA (CHINA) Investment CO., LTD., Beijing, China;University of Science and Technology of China, Hefei, China
Venue:
International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction
Year:
2010

Citing 16
Cited 1

Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video

IEEE Transactions on Pattern Analysis and Machine Intelligence
An Approach Based on Phonemes to Large Vocabulary Chinese Sign Language Recognition

FGR '02 Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition
ASL Recognition Based on a Coupling Between HMMs and 3D Motion Analysis

ICCV '98 Proceedings of the Sixth International Conference on Computer Vision
Using Multiple Sensors for Mobile Sign Language Recognition

ISWC '03 Proceedings of the 7th IEEE International Symposium on Wearable Computers
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
Large vocabulary sign language recognition based on hierarchical decision trees

Proceedings of the 5th international conference on Multimodal interfaces
Enabling fast and effortless customisation in accelerometer based gesture interaction

Proceedings of the 3rd international conference on Mobile and ubiquitous multimedia
Toward subtle intimate interfaces for mobile devices using an EMG controller

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning

IEEE Transactions on Pattern Analysis and Machine Intelligence
Recognition of sign language subwords based on boosted hidden Markov models

ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
Demonstrating the feasibility of using forearm electromyography for muscle-computer interfaces

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Hand gesture recognition and virtual game control based on 3D accelerometer and EMG sensors

Proceedings of the 14th international conference on Intelligent user interfaces
Hand Gesture Recognition Research Based on Surface EMG Sensors and 2D-accelerometers

ISWC '07 Proceedings of the 2007 11th IEEE International Symposium on Wearable Computers
A framework for continuous multimodal sign language recognition

Proceedings of the 2009 international conference on Multimodal interfaces
Gesture Recognition: A Survey

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
Robust coding schemes for indexing and retrieval from large face databases

IEEE Transactions on Image Processing

Gesture vs. gesticulation: a test protocol

HCI'13 Proceedings of the 15th international conference on Human-Computer Interaction: interaction modalities and techniques - Volume Part IV

Quantified Score

Hi-index	0.00

Visualization

Abstract

Sign language recognition (SLR) not only facilitates the communication between the deaf and hearing society, but also serves as a good basis for the development of gesture-based human-computer interaction (HCI). In this paper, the portable input devices based on accelerometers and surface electromyography (EMG) sensors worn on the forearm are presented, and an effective fusion strategy for combination of multi-sensor and multi-channel information is proposed to automatically recognize sign language at the subword classification level. Experimental results on the recognition of 121 frequently used Chinese sign language subwords demonstrate the feasibility of developing SLR system based on the presented portable input devices and that our proposed information fusion method is effective for automatic SLR. Our study will promote the realization of practical sign language recognizer and multimodal human-computer interfaces.