Sentence lipreading using hidden Markov model with integrated grammar

  • Authors:
  • Keren Yu;Xiaoyi Jiang;Horst Bunke

  • Affiliations:
  • Univ. of Bern, Bern, Switzerland;Univ. of Bern, Bern, Switzerland;Univ. of Bern, Bern, Switzerland

  • Venue:
  • Hidden Markov models
  • Year:
  • 2001

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper, we describe a systematic approach to the lipreading of whole sentences. A vocabulary of elementary words is considered. Based on the vocabulary, we define a grammar that generates a set of legal sentences. Our lipreading approach is based on a combination of the grammar with hidden Markov models (HMMs). Two different experiments were conducted. In the first experiment a set of e-mail commands is considered, while the set of sentences in the second experiment is given by all English integer numbers up to one million. Both experiments showed promising results, regarding the difficulty of the considered task.