Automatic music video generation based on temporal pattern analysis

  • Authors:
  • Xian-Sheng HUA;Lie LU;Hong-Jiang ZHANG

  • Affiliations:
  • Microsoft Research Asia;Microsoft Research Asia;Microsoft Research Asia

  • Venue:
  • Proceedings of the 12th annual ACM international conference on Multimedia
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

Music video (MV) is a short film meant to present a visual representation of a popular music song. In this paper, we present a system that automatically generates MV-like videos from personal home videos based on observations that generally there are obvious repetitive visual and aural patterns in MVs. Based on a set of video and music analysis algorithms, the automatic music video (AMV) generation system automatically extracts temporal structures of the video and music, as well as repetitive patterns in the music. And then, according to the structure and patterns, a set of highlight segments from the raw home video footage are selected, aiming at matching the visual content with the aural structure and pattern. And last, the output music video is rendered by connecting the selected highlight video segments with appropriate transition effects, accompanied with the music. Experiments show that the results are compelling and promising.