The image processing handbook (2nd ed.)
The image processing handbook (2nd ed.)
Active shape models—their training and application
Computer Vision and Image Understanding
3D lip shapes from video: a combined physical-statistical model
Speech Communication - Special issue on auditory-visual speech processing
ECCV '98 Proceedings of the 5th European Conference on Computer Vision-Volume II - Volume II
Real-Time Stereo Tracking for Head Pose and Gaze Estimation
FG '00 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000
Adaptive bimodal sensor fusion for automatic speechreading
ICASSP '96 Proceedings of the Acoustics, Speech, and Signal Processing, 1996. on Conference Proceedings., 1996 IEEE International Conference - Volume 02
| Hi-index | 0.01 |
One of the advantages of multimodal HCI technology is the performance improvement that can be gained over conventional single-modality technology by employing complementary sensors in different modalities. Such information is particular useful in practical, real-world applications where the application's performance must be robust against all kinds of noise. An example is the domain of automatic speech recognition (ASR). Traditionally, ASR systems only use information from the audio modality. In the presence of acoustic noise, the performance drops quickly. However, it can and has been shown that incorporating additional visual speech information from the video modality improves the performance significantly, so that AV ASR systems can be employed in applications areas where audio-only ASR systems would fail, thus opening new application areas for ASR technology. In thispaper, a non-intrusive (no artificial markers), real-time 3D lip tracking system is presented as well as its application to AV ASR. The multivariate statistical analysis 'co-inertia analysis' is also shown, which offers improved numerical stability over other multivariate analyses even for small sample sizes.