Modelling and recognition of signed expressions using subunits obtained by data---driven approach

  • Authors:
  • Mariusz Oszust;Marian Wysocki

  • Affiliations:
  • Department of Computer and Control Engineering, Rzeszow University of Technology, Rzeszow, Poland;Department of Computer and Control Engineering, Rzeszow University of Technology, Rzeszow, Poland

  • Venue:
  • AIMSA'12 Proceedings of the 15th international conference on Artificial Intelligence: methodology, systems, and applications
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

The paper considers automatic vision based modelling and recognition of sign language expressions using smaller units than words. Modelling gestures with subunits is similar to modelling speech by means of phonemes. To define the subunits a data---driven procedure is proposed. The procedure consists in partitioning time series of feature vectors obtained from video material into subsequences which form homogeneous clusters. The cut points are determined by an optimisation procedure based on quality assessment of the resulting clusters. Then subunits are selected in two ways: as clusters' representatives or as hidden Markov models of clusters. These two approaches result in differences in classifier design. Details of the solution and results of experiments on a database of 101 Polish words and 35 sentences used at the doctor's and in the post office are given. Our subunit---based classifiers outperform their whole---word---based counterpart, which is particularly evident when new expressions are recognised on the basis of a small number of examples.