Automatic lip contour extraction from color images

Authors:
S. L. Wang;W. H. Lau;S. H. Leung
Affiliations:
Department of Computer Engineering and Information Technology, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon Tong, Hong Kong, China;Department of Computer Engineering and Information Technology, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon Tong, Hong Kong, China;Department of Electronic Engineering, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon Tong, Hong Kong, China
Venue:
Pattern Recognition
Year:
2004

Citing 9
Cited 6

Active shape models—their training and application

Computer Vision and Image Understanding
An Approach to Statistical Lip Modelling for Speaker Identification via Chromatic Feature Extraction

ICPR '98 Proceedings of the 14th International Conference on Pattern Recognition-Volume 1 - Volume 1
Large Vocabulary Audio-Visual Speech Recognition Using Active Shape Models

ICPR '00 Proceedings of the International Conference on Pattern Recognition - Volume 3
Lip Detection and Tracking

ICIAP '01 Proceedings of the 11th International Conference on Image Analysis and Processing
Visual speech recognition using active shape models and hidden Markov models

ICASSP '96 Proceedings of the Acoustics, Speech, and Signal Processing, 1996. on Conference Proceedings., 1996 IEEE International Conference - Volume 02
A new optimization procedure for extracting the point-based lip contour using active shape model

ICASSP '01 Proceedings of the Acoustics, Speech, and Signal Processing, 2001. on IEEE International Conference - Volume 03
A review of speech-based bimodal recognition

IEEE Transactions on Multimedia
Segmentation of color lip images by spatial fuzzy clustering

IEEE Transactions on Fuzzy Systems
Snakes, shapes, and gradient vector flow

IEEE Transactions on Image Processing

Real-time robot manipulation using mouth gestures in facial video sequences

BVAI'07 Proceedings of the 2nd international conference on Advances in brain, vision and artificial intelligence
Vowel recognition by using the combination of Haar wavelet and neural network

KES'10 Proceedings of the 14th international conference on Knowledge-based and intelligent information and engineering systems: Part I
Lip reading of hearing impaired persons using HMM

Expert Systems with Applications: An International Journal
A local region based approach to lip tracking

Pattern Recognition
Automatic detailed localization of facial features

IEA/AIE'12 Proceedings of the 25th international conference on Industrial Engineering and Other Applications of Applied Intelligent Systems: advanced research in applied artificial intelligence
Lip segmentation and tracking under MAP-MRF framework with unknown segment number

Neurocomputing

Quantified Score

Hi-index	0.01

Visualization

Abstract

Visual information from lip shapes and movements helps improve the accuracy and robustness of a speech recognition system. In this paper, a new region-based lip contour extraction algorithm that combines the merits of the point-based model and the parametric model is presented. Our algorithm uses a 16-point lip model to describe the lip contour. Given a robust probability map of the color lip image generated by the FCMS (fuzzy clustering method incorporating shape function) algorithm, a region-based cost function that maximizes the joint probability of the lip and non-lip region can be established. Then an iterative point-driven optimization procedure has been developed to fit the lip model to the probability map. In each iteration, the adjustment of the 16 lip points is governed by three pieces of quadratic curves that constrain the points to form a physical lip shape. Experiments show that the proposed approach provides satisfactory results for 5000 unadorned lip images of over 20 individuals. A real-time lip contour extraction system has also been implemented.