Feature fusion and redundancy pruning for rush video summarization

  • Authors:

  • Jim Kleban;Anindya Sarkar;Emily Moxley;Stephen Mangiat;Swapna Joshi;Thomas Kuo;B. S. Manjunath

  • Affiliations:

  • University of California: Santa Barbara, Santa Barbara, CA;University of California: Santa Barbara, Santa Barbara, CA;University of California: Santa Barbara, Santa Barbara, CA;University of California: Santa Barbara, Santa Barbara, CA;University of California: Santa Barbara, Santa Barbara, CA;University of California: Santa Barbara, Santa Barbara, CA;University of California: Santa Barbara, Santa Barbara, CA

  • Venue:
  • Proceedings of the international workshop on TRECVID video summarization
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper presents a video summarization technique for rushes that employs high-level feature fusion to identify segments for inclusion. It aims to capture distinct video events using a variety of features: k-means based weighting, speech, camera motion, significant differences in HSV color space, and a dynamic time warping (DTW) based feature that suppresses repeated scenes. The feature functions are used to drive a weighted k-means based clustering to identify visually distinct, important segments that constitute the final summary. The optimal weights corresponding to the individual features are obtained using a gradient descent algorithm that maximizes the recall of ground truth events from representative training videos. Analysis reveals a lengthy computation time but high quality results (60% average recall over 42 test videos) as based on manually-judged inclusion ofdistinct shots. The summaries were judged relatively easy to view and had an average amount of redundancy.