Refining video annotation by exploiting inter-shot context

Authors:
Jian Yi;Yuxin Peng;Jianguo Xiao
Affiliations:
Institute of Computer Science and Technology, Peking University, Beijing 100871, China, Beijing, China;Institute of Computer Science and Technology, Peking University, Beijing 100871, China, Beijing, China;Institute of Computer Science and Technology, Peking University, Beijing 100871, China, Beijing, China
Venue:
Proceedings of the international conference on Multimedia
Year:
2010

Citing 7
Cited 1

Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Evaluation campaigns and TRECVid

MIR '06 Proceedings of the 8th ACM international workshop on Multimedia information retrieval
Estimating average precision with incomplete and imperfect judgments

CIKM '06 Proceedings of the 15th ACM international conference on Information and knowledge management
Towards optimal bag-of-features for object categorization and semantic video retrieval

Proceedings of the 6th ACM international conference on Image and video retrieval
Correlative multilabel video annotation with temporal kernels

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
Multi-cue fusion for semantic video indexing

MM '08 Proceedings of the 16th ACM international conference on Multimedia
Association and Temporal Rule Mining for Post-Filtering of Semantic Concept Detection in Video

IEEE Transactions on Multimedia

Mining concept relationship in temporal context for effective video annotation

MM '11 Proceedings of the 19th ACM international conference on Multimedia

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper proposes a new approach to refine video annotation by exploiting the inter-shot context. Our method is mainly novel in two ways. On one hand, to refine annotation result of the target concept, we model the sequence of shots in video as a conditional random field with chain structure. In this way, we can capture different kinds of concept relationships in inter-shot context to improve the annotation accuracy. On the other hand, to exploit inter-shot context for the target concept, we classify shots into different types according to their correlation to the target concept, which will be used to represent different kinds of concept relationships in inter-shot context. Experiments on the widely used TRECVID 2006 data set show that our method is effective for refining video annotation, achieving a significant performance improvement over several state of the art methods.