Educational violin transcription by fusing multimedia streams
Proceedings of the international workshop on Educational multimedia and multimedia education
Hybrid fuzzy HMM system for Arabic connectionist speech recognition
ISPRA'06 Proceedings of the 5th WSEAS International Conference on Signal Processing, Robotics and Automation
The use of articulator motion information in automatic speech segmentation
Speech Communication
Automatic visual feature extraction for mandarin audio-visual speech recognition
SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
An intelligent multimedia E-learning system for pronunciations
IEA/AIE'07 Proceedings of the 20th international conference on Industrial, engineering, and other applications of applied intelligent systems
Bimodal automatic speech segmentation based on audio and visual information fusion
Speech Communication
ASR based on the analasys of the short-melfrequencycepstra time transform
MICAI'05 Proceedings of the 4th Mexican international conference on Advances in Artificial Intelligence
LUI: lip in multimodal mobile GUI interaction
Proceedings of the 14th ACM international conference on Multimodal interaction
| Hi-index | 0.00 |
Audio-visual speech recognition employing both acoustic and visual speech information is a novel extension of acoustic speech recognition and it significantly improves the recognition accuracy in noisy environments. Although various audio-visual speech-recognition systems have been developed, a rigorous and detailed comparison of the potential geometric visual features from speakers' faces is essential. Thus, in this paper the geometric visual features are compared and analyzed rigorously for their importance in audio-visual speech recognition. Experimental results show that among the geometric visual features analyzed, lip vertical aperture is the most relevant; and the visual feature vector formed by vertical and horizontal lip apertures and the first-order derivative of the lip corner angle leads to the best recognition results. Speech signals are modeled by hidden Markov models (HMMs) and using the optimized HMMs and geometric visual features the accuracy of acoustic-only, visual-only, and audio-visual speech recognition methods are compared. The audio-visual speech recognition scheme has a much improved recognition accuracy compared to acoustic-only and visual-only speech recognition especially at high noise levels. The experimental results showed that a set of as few as three labial geometric features are sufficient to improve the recognition rate by as much as 20% (from 62%, with acoustic-only information, to 82%, with audio-visual information at a signal-to-noise ratio of 0 dB).