TRECVID: evaluating the effectiveness of information retrieval tasks on digital video
Proceedings of the 12th annual ACM international conference on Multimedia
Correlative multi-label video annotation
Proceedings of the 15th international conference on Multimedia
Cross-domain video concept detection using adaptive svms
Proceedings of the 15th international conference on Multimedia
A framework for classifier adaptation and its applications in concept detection
MIR '08 Proceedings of the 1st ACM international conference on Multimedia information retrieval
Foundations and Trends in Information Retrieval
Creating a web-scale video collection for research
WSMC '09 Proceedings of the 1st workshop on Web-scale multimedia corpus
Concept detectors: how good is good enough?
MM '09 Proceedings of the 17th ACM international conference on Multimedia
Image annotation using clickthrough data
Proceedings of the ACM International Conference on Image and Video Retrieval
Personalized online video recommendation by neighborhood score propagation based global ranking
Proceedings of the First International Conference on Internet Multimedia Computing and Service
New trends and ideas in visual concept detection: the MIR flickr retrieval evaluation initiative
Proceedings of the international conference on Multimedia information retrieval
Learning automatic concept detectors from online video
Computer Vision and Image Understanding
An improved fusion method based on adaboost algorithm for semantic concept extraction
ICIMCS '10 Proceedings of the Second International Conference on Internet Multimedia Computing and Service
Lost in binarization: query-adaptive ranking for similar image search with compact codes
Proceedings of the 1st ACM International Conference on Multimedia Retrieval
Reliability and effectiveness of clickthrough data for automatic image annotation
Multimedia Tools and Applications
Robust Video Content Analysis via Transductive Learning
ACM Transactions on Intelligent Systems and Technology (TIST)
Efficient spatio-temporal edge descriptor
MMM'12 Proceedings of the 18th international conference on Advances in Multimedia Modeling
Linking visual concept detection with viewer demographics
Proceedings of the 2nd ACM International Conference on Multimedia Retrieval
Simulating the future of concept-based video retrieval under improved detector performance
Multimedia Tools and Applications
Proceedings of the 3rd ACM conference on International conference on multimedia retrieval
The uncertain representation ranking framework for concept-based video retrieval
Information Retrieval
Hi-index | 0.00 |
Great effort has been made to improve video concept detection and continuous progress has been reported. With the current evaluation method being confined to carefully annotated domains and thus quite forgiving, the reliability of the state-of-the-art concept classifiers remains in question. Adopting a more rigorous evaluation approach, we find that most concept classifiers built using the mainstream approach are unreliable because they generalize poorly to domains other than their training domain. Moreover, evidences show that SVM-based concept classifiers learn little beyond memorizing most of the positive training data, and behave close to memory-based models such as kNN indicated by comparable performance between the two models. Examining the properties of the reliable concept classifiers, we find that the classifiers of frequent concepts, "bloated" classifiers, and classifiers capable of learning the pattern of data, tend to be more reliable. This paper contributes to a better understanding of concept detection, suggests heuristics to identify reliable concept classifiers, and discusses solutions to improving concept detection reliability.