Client and speech detection system for intelligent infokiosk

Authors:
Andrey Ronzhin;Alexey Karpov;Irina Kipyatkova;Miloý Železný
Affiliations:
St. Petersburg Institute for Informatics and Automation of RAS, St. Petersburg, Russia;St. Petersburg Institute for Informatics and Automation of RAS, St. Petersburg, Russia;St. Petersburg Institute for Informatics and Automation of RAS, St. Petersburg, Russia;University of West Bohemia, Pilsen, Czech Republic
Venue:
TSD'10 Proceedings of the 13th international conference on Text, speech and dialogue
Year:
2010

Citing 4
Cited 1

Multimodal multispeaker probabilistic tracking in meetings

ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
MATCHKiosk: a multimodal interactive city guide

ACLdemo '04 Proceedings of the ACL 2004 on Interactive poster and demonstration sessions
MIKI: a speech enabled intelligent kiosk

IVA'06 Proceedings of the 6th international conference on Intelligent Virtual Agents
Boosting-Based Multimodal Speaker Detection for Distributed Meeting Videos

IEEE Transactions on Multimedia

An assistive bi-modal user interface integrating multi-channel speech recognition and computer vision

HCII'11 Proceedings of the 14th international conference on Human-computer interaction: interaction techniques and environments - Volume Part II

Quantified Score

Hi-index	0.00

Visualization

Abstract

Timely attraction of a client and detection of his/her speech message in real noisy conditions are main difficulties at deployment of speech and multimodal interfaces in information kiosks. Combination of sound source localization, voice activity and face detection technologies allowed to determine client mouth coordinates and extract boundaries of speech signal appeared in the kiosk dialogue area. Talking head model based on audio-visual speech synthesis immediately greets the client, when her face is captured in the videomonitoring area, in order to attract him/her to the information service before leaving the interaction area. Client's face tracking is also used for turning the talking head in direction to the client that significantly improves the naturalness of interaction. The developed infokiosk set in the institute hall provides information about structure and staff of laboratories. Statistics of human-kiosk interaction is accumulated within last six months in 2009.