Extracting Pointing Object with Demonstrative Speech Phrase for Remote Transcription in Lecture

Authors:
Yoshinori Takeuchi;Ken Saito;Ayaka Ito;Noboru Ohnishi;Shigeyoshi Iizuka;Shinya Nakajima
Affiliations:
Information and Communications Headquarters, Nagoya University, Nagoya, Japan 464-8603 and Bio-Mimetic Control Research Center, RIKEN, Nagoya, Japan 463-0003;Graduate School of Information Science, Nagoya University, Nagoya, Japan 464-8603;Graduate School of Information Science, Nagoya University, Nagoya, Japan 464-8603;Graduate School of Information Science, Nagoya University, Nagoya, Japan 464-8603;NTT Cyber Solutions Laboratories,;School of Science and Engineering, Kokushikan University,
Venue:
ICCHP '08 Proceedings of the 11th international conference on Computers Helping People with Special Needs
Year:
2008

Citing 4
Cited 2

Passive capture and structuring of lectures

MULTIMEDIA '99 Proceedings of the seventh ACM international conference on Multimedia (Part 1)
Shooting the Lecture Scene Using Computer-controlled Cameras Based on Situation Understanding and Evaluation of Video Images

ICPR '04 Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 1 - Volume 01
Camera control for remote transcription system

CGIM '08 Proceedings of the Tenth IASTED International Conference on Computer Graphics and Imaging
Communication supporting system in a classroom environment for the hearing impaired

ICCHP'06 Proceedings of the 10th international conference on Computers Helping People with Special Needs

Extraction of displayed objects corresponding to demonstrative words for use in remote transcription

ICCHP'10 Proceedings of the 12th international conference on Computers helping people with special needs
A system for matching mathematical formulas spoken during a lecture with those displayed on the screen for use in remote transcription

ICCHP'12 Proceedings of the 13th international conference on Computers Helping People with Special Needs - Volume Part I

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this research, we propose a system to extract the object which is pointed by the instructor with a demonstrative speech phrase. The system extracts audio features from a speech signals of the instructor and calculates the likelihood of the demonstrative speech phrase. The system also extracts video features from the video camera and calculates the likelihood of the pointing gesture. Then, the system combines the likelihood of audio and video features to decide whether the instructor points the object with demonstrative speech phrase. We conducted an experiment using a real lecture, which has shown the effectiveness of the proposed method.