Keyframe Extraction Using Local Visual Semantics in the Form of a Region Thesaurus

• Authors: Evaggelos Spyrou;Yannis Avrithis
• Venue: SMAP '07 Proceedings of the Second International Workshop on Semantic Media Adaptation and Personalization
• Year: 2007

This paper presents an approach for efficient keyframe extraction, using local semantics in form of a region the- saurus. More specifically, certain MPEG-7 color and tex- ture features are locally extracted from keyframe regions. Then, using a hierarchical clustering approach a local re- gion thesaurus is constructed to facilitate the description of each frame in terms of higher semantic features. The the- saurus consists of the most common region types that are encountered within the video shot, along with their syn- onyms. These region types carry semantic information. Each keyframe is represented by a vector consisting of the degrees of confidence of the existence of all region types within this shot. Using this keyframe representation, the most representative keyframe is then selected for each shot. Where a single keyframe is not adequate, using the same al- gorithm and exploiting the presence of the region types of the visual thesaurus, more keyframes are extracted.