Lip reading of hearing impaired persons using HMM

Authors:
N. Puviarasan;S. Palanivel
Affiliations:
Department of Computer Science and Engineering, Annamalai University, Annamalainagar 608 002, India;Department of Computer Science and Engineering, Annamalai University, Annamalainagar 608 002, India
Venue:
Expert Systems with Applications: An International Journal
Year:
2011

Citing 6
Cited 2

Speaker-independent 3D face synthesis driven by speech and text

Signal Processing - Fractional calculus applications in signals and systems
The application of extension neuro-network on computer-assisted lip-reading recognition for hearing impaired

Expert Systems with Applications: An International Journal
Facial feature extraction using complex dual-tree wavelet transform

Computer Vision and Image Understanding
Block-based motion estimation analysis for lip reading user authentication systems

WSEAS Transactions on Information Science and Applications
Wrapping snakes for improved lip segmentation

ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
Automatic lip contour extraction from color images

Pattern Recognition

Integration of face detection and user identification with visual speech recognition

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part V
Principal Component Analysis of the start-up transient and Hidden Markov Modeling for broken rotor bar fault diagnosis in asynchronous machines

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	12.05

Visualization

Abstract

This paper describes a method for lip reading of hearing impaired persons. The term lip reading refers to recognizing the spoken words using visual speech information such as lip movements. The visual speech video of the hearing impaired person is given as input to the face detection module for detecting the face region. The region of the mouth is determined relative to the face region. The mouth images are used for feature extraction. The features are extracted using discrete cosine transform (DCT) and discrete wavelet transform (DWT). Then, these features are applied separately as inputs to the hidden markov model (HMM) for recognizing the visual speech. To understand the visual speech of hearing impaired person in cash collection counters, 33 words are chosen. For each word, 20 samples are collected for training the HMM model and another five samples are used for testing the model. The experimental results show that the method gives the performance of 91.0% for the DCT based lip features and 97.0% for DWT based lip features.