Multimodal error correction for continuous handwriting recognition in pen-based user interfaces

Authors:
Xugang Wang;Junfeng Li;Xiang Ao;Gang Wang;Guozhong Dai
Affiliations:
Chinese Academic of Science, Beijing, China;Chinese Academic of Science, Beijing, China;Chinese Academic of Science, Beijing, China;France Telecom Research & Development Beijing, Beijing, China;Chinese Academic of Science, Beijing, China
Venue:
Proceedings of the 11th international conference on Intelligent user interfaces
Year:
2006

Citing 8
Cited 4

Decline in accuracy of automatic speech recognition as a function of time on task: fatigue or voice drift?

International Journal of Man-Machine Studies
Integration and synchronization of input modes during multimodal human-computer interaction

Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
Taming recognition errors with a multimodal interface

Communications of the ACM
Multimodal error correction for speech user interfaces

ACM Transactions on Computer-Human Interaction (TOCHI)
Oops! silly me! errors in a handwriting recognition-based text entry interface for children

Proceedings of the second Nordic conference on Human-computer interaction
An investigation into the use of error recovery dialogues in a user interface management system for speech recognition

INTERACT '90 Proceedings of the IFIP TC13 Third Interational Conference on Human-Computer Interaction
Finite-state multimodal parsing and understanding

COLING '00 Proceedings of the 18th conference on Computational linguistics - Volume 1
Multimodal integration: a biological view

IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2

Crossmodal error correction of continuous handwriting recognition by speech

Proceedings of the 12th international conference on Intelligent user interfaces
Multimodal Chinese text entry with speech and keypad on mobile devices

Proceedings of the 13th international conference on Intelligent user interfaces
Understanding, Manipulating and Searching Hand-Drawn Concept Maps

ACM Transactions on Intelligent Systems and Technology (TIST)
Aiding human discovery of handwriting recognition errors

Proceedings of the 15th ACM on International conference on multimodal interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we describe a multimodal error correction mechanism. It allows the user to correct errors in continuous handwriting recognition naturally by simultaneously using pen gesture and speech. A multimodal fusion algorithm is designed to enhance recognition accuracies of handwriting and speech through cross-modal influence. We have performed preliminary evaluation experiments and the results show that this multimodal mechanism can efficiently correct the errors in continuous handwriting recognition.