“Put-that-there”: Voice and gesture at the graphics interface
SIGGRAPH '80 Proceedings of the 7th annual conference on Computer graphics and interactive techniques
A Parallel Algorithm for Dynamic Gesture Tracking
RATFG-RTS '99 Proceedings of the International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems
Deterministic parsing of syntactic non-fluencies
ACL '83 Proceedings of the 21st annual meeting on Association for Computational Linguistics
A speech-first model for repair detection and correction
ACL '93 Proceedings of the 31st annual meeting on Association for Computational Linguistics
Multimodal model integration for sentence unit detection
Proceedings of the 6th international conference on Multimodal interfaces
Utilizing gestures to better understand dynamic structure of human communication
Proceedings of the 6th international conference on Multimodal interfaces
Incorporating gesture and gaze into multimodal models of human-to-human communication
NAACL-DocConsortium '06 Proceedings of the 2006 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: companion volume: doctoral consortium
The catchment feature model: a device for multimodal fusion and a bridge between signal and sense
EURASIP Journal on Applied Signal Processing
HLT-NAACL-Short '04 Proceedings of HLT-NAACL 2004: Short Papers
Cognitive principles in robust multimodal interpretation
Journal of Artificial Intelligence Research
TreeHeaven: a table game using vision-based gesture recognition
Proceedings of the 2011 ACM symposium on The role of design in UbiComp research & practice
| Hi-index | 0.00 |
Speech and gesture are two primary modes used in natural human communication; hence, they are important inputs for a multimodal interface to process. One of the challenges for multimodal interfaces is to accurately recognize the words in spontaneous speech. This is partly due to the presence of speech repairs, which seriously degrade the accuracy of current speech recognition systems. Based on the assumption that speech and gesture arise from same thought process, we would expect to find patterns of gesture that co-occur with speech repairs that can be exploited by a multimodal processing system to more effectively process spontaneous speech.To evaluate this hypothesis, we have conducted a measurement study of gesture and speech repair data extracted from videotapes of natural dialogs. Although we have found that gestures do not always co-occur with speech repairs, we observed that modification gesture patterns have a high correlation with content replacement speech repairs, but rarely occur with content repetitions. These results suggest that gesture patterns can help us to classify different types of speech repairs in order to correct them more accurately [6].