Audio-visual tracking for natural interactivity
MULTIMEDIA '99 Proceedings of the seventh ACM international conference on Multimedia (Part 1)
Performance-driven hand-drawn animation
NPAR '00 Proceedings of the 1st international symposium on Non-photorealistic animation and rendering
A Robust Algorithm for 3D Head Pose Estimation
ICPR '98 Proceedings of the 14th International Conference on Pattern Recognition-Volume 2 - Volume 2
A novel face-tracking mouth controller and its application to interacting with bioacoustic models
NIME '04 Proceedings of the 2004 conference on New interfaces for musical expression
Performance-driven hand-drawn animation
ACM SIGGRAPH 2006 Courses
Learning in intelligent embedded systems
WOES'99 Proceedings of the Workshop on Embedded Systems on Workshop on Embedded Systems
Audio-visual speech recognition using MPEG-4 compliant visual features
EURASIP Journal on Applied Signal Processing
| Hi-index | 0.00 |
The robust acquisition of facial features needed for visual speech processing is fraught with difficulties which greatly increase the complexity of the machine vision system. This system must extract the inner lip contour from facial images with variations in pose, lighting, and facial hair. This paper describes a face feature acquisition system with robust performance in the presence of extreme lighting variations and moderate variations in pose. Furthermore, system performance is not degraded by facial hair or glasses. To find the position of a face reliably we search the whole image for facial features. These features are then combined and tests are applied, to determine whether any such combination actually belongs to a face. In order to find where the lips are, other features of the face, such as the eyes, must be located as well. Without this information it is difficult to reliably find the mouth in a complex image. Just the mouth by itself is easily missed or other elements in the image can be mistaken for a mouth. If camera position can be constrained to allow the nostrils to be viewed, then nostril tracking is used to both reduce computation and provide additional robustness. Once the nostrils are tracked from frame to frame using a tracking window the mouth area can be isolated and normalized for scale and rotation. A mouth detail analysis procedure is then used to estimate the inner lip contour and teeth and tongue regions. The inner lip contour and head movements are then mapped to synthetic face parameters to generate a graphical talking head synchronized with the original human voice. This information can also be used as the basis for visual speech features in an automatic speechreading system. Similar features were used in our previous automatic speechreading systems.