Automatic music video generation: cross matching of music and image

  • Authors:

  • Xixuan Wu;Bing Xu;Yu Qiao;Xiaoou Tang

  • Affiliations:

  • The Chinese University of Hong Kong, Hong Kong, Hong Kong;The Chinese University of Hong Kong, Hong Kong, Hong Kong;Chinese Academy of Sciences, Shenzhen, China;The Chinese University of Hong Kong, Hong Kong, Hong Kong

  • Venue:
  • Proceedings of the 20th ACM international conference on Multimedia
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Music and image are two most popular media on the Internet. Human perception of music and image are highly correlated. Music video is one of such products, in which music and image are complement to each other. In this paper, we present a system which can automatically generate music video for a given song. The challenge of such system comes from how to select relative images and align them with the song. This paper deals with this challenge by leveraging lyrics (if exists) and the semantic similarity between music and image. We retrieve related image in internet with lyrics keyword as query and use a learning based method to estimate a semantic score between an image and a music segment. Finally we construct a music video after quality filtering and refinement. Our system also allows users to upload their images and re-pick recommended images to personalize the music video.