BLEU: a method for automatic evaluation of machine translation
ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Automatic evaluation of summaries using N-gram co-occurrence statistics
NAACL '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1
Video abstraction: A systematic review and classification
ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
Comparison of multiepisode video summarization algorithms
EURASIP Journal on Applied Signal Processing
The trecvid 2007 BBC rushes summarization evaluation pilot
Proceedings of the international workshop on TRECVID video summarization
Video summarisation: A conceptual framework and survey of the state of the art
Journal of Visual Communication and Image Representation
Automatic evaluation method for rushes summary content
ICME'09 Proceedings of the 2009 IEEE international conference on Multimedia and Expo
3rd international workshop on automated information extraction in media production
Proceedings of the international conference on Multimedia
Video Segmentation and Structuring for Indexing Applications
International Journal of Multimedia Data Engineering & Management
| Hi-index | 0.00 |
Video Summarization has become an important tool for multimedia information processing, but the automatic evaluation of a video summarization system remains a challenge. A major issue is that an ideal "best" summary does not exist, although people can easily distinguish "good" from "bad" summaries. A similar situation arise in machine translation and text summarization, where specific automatic procedures, respectively BLEU and ROUGE, evaluate the quality of a candidate by comparing its local similarities with several human-generated references. These procedures are now routinely used in various benchmarks. In this paper, we extend this idea to the video domain and propose the VERT (Video Evaluation by Relevant Threshold) algorithm to automatically evaluate the quality of video summaries. VERT mimics the theories of BLEU and ROUGE, and counts the weighted number of overlapping selected units between the computer-generated video summary and several human-made references. Several variants of VERT are suggested and compared.