Sort-merge feature selection and fusion methods for classification of unstructured video

Authors:
Mitchell J. Morris;John R. Kender
Affiliations:
Department of Computer Science, Columbia University;Department of Computer Science, Columbia University
Venue:
ICME'09 Proceedings of the 2009 IEEE international conference on Multimedia and Expo
Year:
2009

Citing 8
Cited 2

Browsing digital video

Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Color Texture Recognition in Video Sequences using Wavelet Covariance Features and Support Vector Machines

EUROMICRO '03 Proceedings of the 29th Conference on EUROMICRO
An introduction to variable and feature selection

The Journal of Machine Learning Research
Fast video segment retrieval by sort-merge feature selection, boundary refinement, and lazy evaluation

Computer Vision and Image Understanding - Special isssue on video retrieval and summarization
Random Subspaces and Subsampling for 2-D Face Recognition

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Benchmarking image and video retrieval: an overview

MIR '06 Proceedings of the 8th ACM international workshop on Multimedia information retrieval
LIBSVM: A library for support vector machines

ACM Transactions on Intelligent Systems and Technology (TIST)
Automatic text detection and tracking in digital video

IEEE Transactions on Image Processing

VastMM-Tag: a semantic tagging browser for unstructured videos

MM '11 Proceedings of the 19th ACM international conference on Multimedia
The appearance of the giant component in descriptor graphs and its application for descriptor selection

CLEF'12 Proceedings of the Third international conference on Information Access Evaluation: multilinguality, multimodality, and visual analytics

Quantified Score

Hi-index	0.00

Visualization

Abstract

We explore the problem of rapid automatic semantic tagging of video frames of unstructured (unedited) videos. We apply the Sort-Merge algorithrn for feature selection on a large (1000) heterogeneous feature set for videos showing lectures, to quickly locate low-level image features most predictive for concepts such as "key frame with text" or "key frame with computer source code". For evaluation, we introduce a "keeper" heuristic for feature retention, which provides a baseline comparison. We then compare early fusion and late fusion of diverse feature types; based on experiments on 12,395 frames, we fmd that in general late fusion offers higher Average Precision accuracy at lower computation cost, compared to early fusion. However, mergers of redundant feature types do not necessarily improve performance over single feature types; exploration of both merged and unmerged performance is necessary.