Multimodal Chinese text entry with speech and keypad on mobile devices

Authors:
Yingying Jiang;Xugang Wang;Feng Tian;Xiang Ao;Guozhong Dai;Hongan Wang
Affiliations:
Chinese Academy of Sciences, Beijing, China;Chinese Academy of Sciences, Beijing, China and Ministry of information Industry Software and Integrated Circuit Promotion Center;Chinese Academy of Sciences, Beijing, China;Ministry of information Industry Software and Integrated Circuit Promotion Center;Chinese Academy of Sciences, Beijing, China;Chinese Academy of Sciences, Beijing, China
Venue:
Proceedings of the 13th international conference on Intelligent user interfaces
Year:
2008

Citing 8
Cited 2

A flexible Chinese character input scheme

UIST '91 Proceedings of the 4th annual ACM symposium on User interface software and technology
Ten myths of multimodal interaction

Communications of the ACM
Chinese input with keyboard and eye-tracking: an anatomical study

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Graphics matter: a case study of mobile phone keypad design for chinese input

CHI '05 Extended Abstracts on Human Factors in Computing Systems
Multimodal error correction for continuous handwriting recognition in pen-based user interfaces

Proceedings of the 11th international conference on Intelligent user interfaces
Speech pen: predictive handwriting based on ambient multimodal recognition

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Using redundant speech and handwriting for learning new vocabulary and understanding abbreviations

Proceedings of the 8th international conference on Multimodal interfaces
Crossmodal error correction of continuous handwriting recognition by speech

Proceedings of the 12th international conference on Intelligent user interfaces

Multi-modal text entry and selection on a mobile device

Proceedings of Graphics Interface 2010
Hype or Ready for Prime Time?: Speech Recognition on Mobile Handheld Devices MASR

International Journal of Handheld Computing Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

Chinese text entry is challenging on mobile devices which rely on keypad input. Entering one character may require many key presses. This paper proposes a multimodal text entry technique for Chinese. In this method, Chinese user can enter Chinese text by simultaneously using the simplified phonemic input method named Jianpin with keypad and speech utterance. The key of the technique is a multimodal fusion algorithm, which synchronizes speech and keypad input and fuses redundant information from two modalities to get the best candidate. A preliminary evaluation shows that users appreciate this technique and it could reduce key presses and enhance the input efficiency.