Audio-visual fusion using bayesian model combination for web video retrieval

Authors:
Vasant Manohar;Stavros Tsakalidis;Pradeep Natarajan;Rohit Prasad;Prem Natarajan
Affiliations:
Raytheon BBN Technologies, Cambridge, MA, USA;Raytheon BBN Technologies, Cambridge, MA, USA;Raytheon BBN Technologies, Cambridge, MA, USA;Raytheon BBN Technologies, Cambridge, MA, USA;Raytheon BBN Technologies, Cambridge, MA, USA
Venue:
MM '11 Proceedings of the 19th ACM international conference on Multimedia
Year:
2011

Citing 8
Cited 1

Automatic partitioning of full-motion video

Multimedia Systems
Distinctive Image Features from Scale-Invariant Keypoints

International Journal of Computer Vision
A Bayesian Hierarchical Model for Learning Natural Scene Categories

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Early versus late fusion in semantic video analysis

Proceedings of the 13th annual ACM international conference on Multimedia
Early versus late fusion in semantic video analysis

Proceedings of the 13th annual ACM international conference on Multimedia
Large-scale multimodal semantic concept detection for consumer video

Proceedings of the international workshop on Workshop on multimedia information retrieval
Speeded-Up Robust Features (SURF)

Computer Vision and Image Understanding
Short-term audio-visual atoms for generic video concept classification

MM '09 Proceedings of the 17th ACM international conference on Multimedia

Video content categorization using the double decomposition

Multimedia Tools and Applications

Quantified Score

Hi-index	0.00

Visualization

Abstract

Combining features from multiple, heterogeneous, audio visual sources can significantly improve retrieval performance in consumer domain videos. However, such videos often contain unrelated overlaid audio content, or have significant camera motion to reliably extract visual features. We present an approach, which overcomes errors in individual feature streams by combining classifiers trained on multiple, heterogeneous feature streams using Bayesian model combination (BAYCOM). We demonstrate our method, by combining low-level audio and visual features, for classification of a large 200 hour web video corpus. The combined models outperform any of the individual features by 10%. Further, BAYCOM consistently outperforms traditional early and late fusion methods.