Visual speech recognition using motion features and hidden Markov models

Authors:
Wai Chee Yau;Dinesh Kant Kumar;Hans Weghorn
Affiliations:
School of Electrical and Computer Engineering, RMIT University, Melbourne, Victoria, Australia;School of Electrical and Computer Engineering, RMIT University, Melbourne, Victoria, Australia;Information Technology, BA-University of Cooperative Education, Stuttgart, Germany
Venue:
CAIP'07 Proceedings of the 12th international conference on Computer analysis of images and patterns
Year:
2007

Citing 6
Cited 4

On Image Analysis by the Methods of Moments

IEEE Transactions on Pattern Analysis and Machine Intelligence
Invariant Image Recognition by Zernike Moments

IEEE Transactions on Pattern Analysis and Machine Intelligence
The Recognition of Human Movement Using Temporal Templates

IEEE Transactions on Pattern Analysis and Machine Intelligence
Recognition of Visual Speech Elements Using Hidden Markov Models

PCM '02 Proceedings of the Third IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing
Visual Speech Recognition Method Using Translation, Scale and Rotation Invariant Features

AVSS '06 Proceedings of the IEEE International Conference on Video and Signal Based Surveillance
Visual model structures and synchrony constraints for audio-visual speech recognition

IEEE Transactions on Audio, Speech, and Language Processing

Private predictions on hidden Markov models

Artificial Intelligence Review
Comparative analysis of lip features for person identification

Proceedings of the 8th International Conference on Frontiers of Information Technology
Robust visual speakingness detection using bi-level HMM

Pattern Recognition
Adaptive Reliability Measure and Optimum Integration Weight for Decision Fusion Audio-visual Speech Recognition

Journal of Signal Processing Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a novel visual speech recognition approach based on motion segmentation and hidden Markov models (HMM). The proposed method identifies utterances from mouth video, without evaluating voice signals. The facial movements in the video data are represented using 2D spatial-temporal templates (STT). The proposed technique combines discrete stationary wavelet transform (SWT) and Zernike moments to extract rotation invariant features from the STTs. HMMs are used as speech classifier to model English phonemes. The preliminary results demonstrate that the proposed technique is suitable for phoneme classification with a high accuracy.