Emotion-based music visualization using photos

  • Authors:

  • Chin-Han Chen;Ming-Fang Weng;Shyh-Kang Jeng;Yung-Yu Chuang

  • Affiliations:

  • Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan;Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan;Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan;Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan

  • Venue:
  • MMM'08 Proceedings of the 14th international conference on Advances in multimedia modeling
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Music players for personal computers are often featured with music visualization by generating animated patterns according to the music's low-level features such as loudness and spectrum. This paper proposes an emotion-based music player which synchronizes visualization (photos) with music based on the emotions evoked by auditory stimulus of music and visual content of visualization. For emotion detection from photos, we collected 398 photos with their emotions annotated by 496 users through the web. With these annotations, a Bayesian classification method is proposed for automatic photo emotion detection. For emotion detection from music, we adopt an existing method. Finally, for composition of music and photos, in addition to matching high-level emotions, we also consider low-level feature harmony and temporal visual coherence. It is formulated as an optimization problem and solved by a greedy algorithm. Subjective evaluation shows emotion-based music visualization enriches users' listening experiences.