Automatic recognition fingerspelling gestures in multiple languages for a communication interface for the disabled

Authors:
A. Alp Kindiroglu;H. Yalcin;O. Aran;M. Hrúz;P. Campr;L. Akarun;A. Karpov
Affiliations:
Bogazici University, Istanbul, Turkey;Bogazici University, Istanbul, Turkey;Idiap Research Institute, Martigny, Switzerland;Faculty of Applied Sciences, Department of Cybernetics, University of West Bohemia, Pilsen, Czech Republic;Faculty of Applied Sciences, Department of Cybernetics, University of West Bohemia, Pilsen, Czech Republic;Bogazici University, Istanbul, Turkey;St.-Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St.-Petersburg, Russia
Venue:
Pattern Recognition and Image Analysis
Year:
2012

Citing 6
Cited 0

New forms of shape invariants from elliptic Fourier descriptors

Pattern Recognition
Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns

IEEE Transactions on Pattern Analysis and Machine Intelligence
Object tracking using CamShift algorithm and multiple quantized feature spaces

VIP '05 Proceedings of the Pan-Sydney area workshop on Visual information processing
Czech text-to-sign speech synthesizer

MLMI'07 Proceedings of the 4th international conference on Machine learning for multimodal interaction
Turkish Broadcast News Transcription and Retrieval

IEEE Transactions on Audio, Speech, and Language Processing
Shape-based hand recognition

IEEE Transactions on Image Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents the design and evaluation of a multi-lingual fingerspelling recognition module that is designed for an information terminal. Through the use of multimodal input and output methods, the information terminal acts as a communication medium between deaf and blind people. The system converts fingerspelled words to speech and vice versa using fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis in Czech, Russian, and Turkish languages. We describe an adaptive skin color based fingersign recognition system with a close to real-time performance and present recognition results on 88 different letters signed by five different signers, using above four hours of training and test videos.