An Improved Fusion Design of Audio-Gesture for Multi-modal HCI Based on Web and WPS

Authors:
Jung-Hyun Kim;Kwang-Seok Hong
Affiliations:
School of Information and Communication Engineering, Sungkyunkwan University, 300, Chunchun-dong, Jangan-gu, Suwon, KyungKi-do, 440-746, Korea;School of Information and Communication Engineering, Sungkyunkwan University, 300, Chunchun-dong, Jangan-gu, Suwon, KyungKi-do, 440-746, Korea
Venue:
ICESS '07 Proceedings of the 3rd international conference on Embedded Software and Systems
Year:
2007

Citing 7
Cited 0

Reasoning about knowledge

Reasoning about knowledge
Fuzzy logic and neural network handbook

Fuzzy logic and neural network handbook
Pattern classification: a unified view of statistical and neural approaches

Pattern classification: a unified view of statistical and neural approaches
Architecture of Multi-modal Dialogue System

TDS '00 Proceedings of the Third International Workshop on Text, Speech and Dialogue
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
Applied Pattern Recognition.

Applied Pattern Recognition.
Hand gesture recognition system using fuzzy algorithm and RDBMS for post PC

FSKD'05 Proceedings of the Second international conference on Fuzzy Systems and Knowledge Discovery - Volume Part II

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper introduces improved fission rule depending on SNNR (Signal Plus Noise to Noise Ratio) and fuzzy value for simultaneous multi-modality, and suggests the Fusion User Interface (hereinafter, FUI) including a synchronization between audio-gesture modalities, based on the embedded KSSL (Korean Standard Sign Language) recognizer using the WPS (Wearable Personal Station for the next generation PC) and Voice-XML. Our approach fuses and recognizes 62 sentential and 152 word language models that are represented by speech and KSSL, then translates recognition results that is fissioned according to a weight decision rule into synthetic speech and visual illustration (graphical display by HMD-Head Mounted Display) in real-time. The experimental results, average recognition rates of the FUI for 62 sentential and 152 word language models were 94.33% and 96.85% in clean environments (e.g. office space), and 92.29% and 92.91% were shown in noisy environments.