An intelligent multimedia E-learning system for pronunciations

Authors:
Wen-Chen Huang;Tsai-Lu Chang-Chien;Hsiu-Pi Lin
Affiliations:
National Kaohsiung First University of Science and Technology, Kaohsiung, Taiwan;National Kaohsiung First University of Science and Technology, Kaohsiung, Taiwan;Chia-Nan University of Pharmacy and Science, Tainan, Taiwan and National Yunlin University of Science and Technology, Yunlin, Taiwan
Venue:
IEA/AIE'07 Proceedings of the 20th international conference on Industrial, engineering, and other applications of applied intelligent systems
Year:
2007

Citing 5
Cited 0

A multimedia framework for second language teaching in self-access environments

Computers & Education
Extraction of Visual Features for Lipreading

IEEE Transactions on Pattern Analysis and Machine Intelligence
Applying a multimedia storytelling website in foreign language learning

Computers & Education
Analysis of lip geometric features for audio-visual speech recognition

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans
Lip image segmentation using fuzzy clustering incorporating an elliptic shape function

IEEE Transactions on Image Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

The proposed system relates to an interactive scoring system for learning a language, in which a means such as a web camera is used to capture the learners lip movements and then a score is given by making a comparison with images stored in the database. The images stored in the database are those previously recorded by a teacher. By means of the scoring system, the learner can identify and rectify pronunciation problems concerning the lips and tongue. The system also records sounds as well as images from the student. The proposed system processes this data with multimedia processing techniques. With regard to the interactive perspective, a user-friendly visual interface was constructed to help learners use the system. The learners can choose the words they want to practice by capturing their lip image sequences and speech. The lip region image sequences are extracted automatically as visual feature parameters. Combining the visual and voice parameters, the proposed system calculates the similarity between a learners and a teachers pronunciation. An evaluation score is suggested by the proposed system through the previous similarity computation. By this learning process, learners can see the corresponding lip movement of both themselves and a teacher, and correct their pronunciation accordingly. The learners can use the proposed system to practice their pronunciation as many times as they like, without troubling the human teacher, and thus they are able to take more control of improving their pronunciation.