Spoken Word Recognition from Side of Face Using Infrared Lip Movement Sensor

Authors:
Takahiro Yoshida;Erika Yamazaki;Seiichiro Hangai
Affiliations:
Department of Electrical Engineering, Tokyo University of Science, Tokyo, Japan 102-0073;Department of Electrical Engineering, Tokyo University of Science, Tokyo, Japan 102-0073;Department of Electrical Engineering, Tokyo University of Science, Tokyo, Japan 102-0073
Venue:
PIT '08 Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems
Year:
2008

Citing 4
Cited 0

Accurate, Real-Time, Unadorned Lip Tracking

ICCV '98 Proceedings of the Sixth International Conference on Computer Vision
Real Time Tracking for 3D Realistic Lip Animation

ICPR '06 Proceedings of the 18th International Conference on Pattern Recognition - Volume 01
Adaptive bimodal sensor fusion for automatic speechreading

ICASSP '96 Proceedings of the Acoustics, Speech, and Signal Processing, 1996. on Conference Proceedings., 1996 IEEE International Conference - Volume 02
The use of temporal speech and lip information for multi-modal speaker identification via multi-stream HMMs

ICASSP '00 Proceedings of the Acoustics, Speech, and Signal Processing, 2000. on IEEE International Conference - Volume 04

Quantified Score

Hi-index	0.00

Visualization

Abstract

In order to realize multimodal speech recognition on a mobile phone, it is necessary to develop a small sensor which enables to measure lip movement with small calculation cost. In the previous study, we have developed a simple infrared lip movement sensor located on the front of mouth and cleared that the possibility of HMM based word recognition with 87.1% recognition rate. However, in practical use, it is difficult to set the sensor in front of mouth. In this paper, we developed a new lip movement sensor which can extract the lip movement from either side of a speaker's face and examine the performance. From experimental results, we have achieved 85.3% speaker independent word recognition rate only with the lip movement from the side sensor.