A new manifold representation for visual speech recognition

  • Authors:

  • Dahai Yu;Ovidiu Ghita;Alistair Sutherland;Paul F. Whelan

  • Affiliations:

  • School of Computing & Electronic Engineering, Vision Systems Group, Dublin City University, Dublin 9, Ireland;School of Computing & Electronic Engineering, Vision Systems Group, Dublin City University, Dublin 9, Ireland;School of Computing & Electronic Engineering, Vision Systems Group, Dublin City University, Dublin 9, Ireland;School of Computing & Electronic Engineering, Vision Systems Group, Dublin City University, Dublin 9, Ireland

  • Venue:
  • CAIP'07 Proceedings of the 12th international conference on Computer analysis of images and patterns
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper, we propose a new manifold representation capable of being applied for visual speech recognition. In this regard, the real time input video data is compressed using Principal Component Analysis (PCA) and the low-dimensional points calculated for each frame define the manifolds. Since the number of frames that from the video sequence is dependent on the word complexity, in order to use these manifolds for visual speech classification it is required to re-sample them into a fixed number of keypoints that are used as input for classification. In this paper two classification schemes, namely the k Nearest Neighbour (kNN) algorithm that is used in conjunction with the two-stage PCA and Hidden-Markov-Model (HMM) classifier are evaluated. The classification results for a group of English words indicate that the proposed approach is able to produce accurate classification results.