Towards a new reading experience via semantic fusion of text and music

  • Authors:
  • Ling Zhuang;Zhenchao Ye;Jiangqin Wu;Feng Zhou;Jian Shao

  • Affiliations:
  • College of Computer Science Zhejiang University, Hang Zhou, China;College of Computer Science Zhejiang University, Hang Zhou, China;College of Computer Science Zhejiang University, Hang Zhou, China;College of Computer Science Zhejiang University, Hang Zhou, China;College of Computer Science Zhejiang University, Hang Zhou, China

  • Venue:
  • Proceedings of the 11th annual international ACM/IEEE joint conference on Digital libraries
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

In CADAL, there preserve a lot of Chinese classical literatures, including graceful prose and verse. These works written in ancient Chinese comparatively are concise in vocabulary and sentence patterns. But they express rich feelings and convey a wealth of information. Although can be explained in modern Chinese, the aesthetic sense in those works disappears. So we aim to illustrate the feeling in these works using Chinese traditional music which is also another part of Chinese culture. This is an interesting and challenging work. In this paper, the correlation between the text and music is studied. A novel approach is proposed to model the latent semantic association underlying the two medium. Based on the correlation model we learned from training data, we can associate a literary work (mainly verse and prose in our digital library) with a few music pieces automatically. When a reader is appreciating a literary work, a piece of background music is playing meanwhile, the information and emotion implied by the work and music blend together. The reader may be immersed into the emotion and obtain aesthetic enjoyment intensively. We implement the proposed method and design experiments to evaluate the performance of it. The experimental result substantiates the feasibility of the proposed approach in this paper.