Visual lip activity detection and speaker detection using mouth region intensities

Authors:
Spyridon Siatras;Nikos Nikolaidis;Michail Krinidis;Ioannis Pitas
Affiliations:
Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece;Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece;Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece;Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece
Venue:
IEEE Transactions on Circuits and Systems for Video Technology
Year:
2009

Citing 7
Cited 3

Array Signal Processing: Concepts and Techniques

Array Signal Processing: Concepts and Techniques
Robust Real-Time Face Detection

International Journal of Computer Vision
Audio-visual intent-to-speak detection for human-computer interaction

ICASSP '00 Proceedings of the Acoustics, Speech, and Signal Processing, 2000. on IEEE International Conference - Volume 04
Audio-visual speech recognition using MPEG-4 compliant visual features

EURASIP Journal on Applied Signal Processing
A support vector machine-based dynamic network for visual speech recognition applications

EURASIP Journal on Applied Signal Processing
Synergy of Lip-Motion and Acoustic Features in Biometric Speech and Speaker Recognition

IEEE Transactions on Computers
Extraction of Audio Features Specific to Speech Production for Multimodal Speaker Detection

IEEE Transactions on Multimedia

Unsupervised temporal segmentation of talking faces using visual cues to improve emotion recognition

ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part I
Robust visual speakingness detection using bi-level HMM

Pattern Recognition
Real-time audio-visual analysis for multiperson videoconferencing

Advances in Multimedia

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this letter, we introduce a novel approach for lip activity detection and speaker detection, using solely visual. information. The main idea in this work is to apply signal detection algorithms to a simple and easily extracted feature from the mouth region. We argue that the increased average value and standard deviation of the number of pixels with low intensities that the mouth region of a speaking person demonstrates can be used as visual cues for detecting visual speech. We then proceed in deriving a statistical algorithm that utilizes this fact for the efficient characterization of visual speech and silence in video sequences. Furthermore, we employ the lip activity detection method in order to determine the active speaker(s) in a multi-person environment.