Active learning in multimedia annotation and retrieval: A survey

Authors:
Meng Wang;Xian-Sheng Hua
Affiliations:
Microsoft Research Asia, Beijing, China;Microsoft Research Asia, Beijing, China
Venue:
ACM Transactions on Intelligent Systems and Technology (TIST)
Year:
2011

Citing 47
Cited 32

Query by committee

COLT '92 Proceedings of the fifth annual workshop on Computational learning theory
Improving Generalization with Active Learning

Machine Learning - Special issue on structured connectionist systems
A maximum entropy approach to natural language processing

Computational Linguistics
Solving the multiple instance problem with axis-parallel rectangles

Artificial Intelligence
Selective Sampling Using the Query by Committee Algorithm

Machine Learning
Combining labeled and unlabeled data with co-training

COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
Content-Based Image Retrieval at the End of the Early Years

IEEE Transactions on Pattern Analysis and Machine Intelligence
Support vector machine active learning for image retrieval

MULTIMEDIA '01 Proceedings of the ninth ACM international conference on Multimedia
Queries and Concept Learning

Machine Learning
Queries and Concept Learning

Machine Learning
Toward Optimal Active Learning through Sampling Estimation of Error Reduction

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Active + Semi-supervised Learning = Robust Multi-View Learning

ICML '02 Proceedings of the Nineteenth International Conference on Machine Learning
Multiple-Instance Learning for Natural Scene Classification

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Support Vector Machine Active Learning with Application sto Text Classification

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Automatically Labeling Video Data Using Multi-class Active Learning

ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
Labeling images with a computer game

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Active learning using pre-clustering

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Mean version space: a new active learning method for content-based image retrieval

Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval
Multimodal concept-dependent active learning for image retrieval

Proceedings of the 12th annual ACM international conference on Multimedia
On the detection of semantic concepts at TRECVID

Proceedings of the 12th annual ACM international conference on Multimedia
A comparison of active classification methods for content-based image retrieval

Proceedings of the 1st international workshop on Computer vision meets databases
A Semi-Supervised Active Learning Framework for Image Retrieval

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
A web-based system for collaborative annotation of large image and video collections: an evaluation and user study

Proceedings of the 13th annual ACM international conference on Multimedia
Putting active learning into multimedia applications: dynamic definition and refinement of concept classifiers

Proceedings of the 13th annual ACM international conference on Multimedia
Semi-automatic video annotation based on active learning with multiple complementary predictors

Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval
Peekaboom: a game for locating objects in images

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Batch mode active learning and its application to medical image classification

ICML '06 Proceedings of the 23rd international conference on Machine learning
Concept boundary detection for speeding up SVMs

ICML '06 Proceedings of the 23rd international conference on Machine learning
Video Annotation by Active Learning and Cluster Tuning

CVPRW '06 Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop
Active learning in very large databases

Multimedia Tools and Applications
Correlative multi-label video annotation

Proceedings of the 15th international conference on Multimedia
A dual coordinate descent method for large-scale linear SVM

Proceedings of the 25th international conference on Machine learning
On multi-view active learning and the combination with semi-supervised learning

Proceedings of the 25th international conference on Machine learning
Real-Time Computerized Annotation of Pictures

IEEE Transactions on Pattern Analysis and Machine Intelligence
Online multi-label active annotation: towards large-scale content-based video search

MM '08 Proceedings of the 16th ACM international conference on Multimedia
Towards Scalable Dataset Construction: An Active Learning Approach

ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
Hybrid Tagging and Browsing Approaches for Efficient Manual Image Annotation

IEEE MultiMedia
Two-Dimensional Multilabel Active Learning with an Efficient Online Adaptation Model for Image Classification

IEEE Transactions on Pattern Analysis and Machine Intelligence
Active learning with statistical models

Journal of Artificial Intelligence Research
Locally non-negative linear structure learning for interactive image retrieval

MM '09 Proceedings of the 17th ACM international conference on Multimedia
Unified video annotation via multigraph learning

IEEE Transactions on Circuits and Systems for Video Technology
Beyond distance measurement: constructing neighborhood similarity for video annotation

IEEE Transactions on Multimedia - Special section on communities and media computing
Multi-view multi-label active learning for image classification

ICME'09 Proceedings of the 2009 IEEE international conference on Multimedia and Expo
Multiple kernel active learning for image classification

ICME'09 Proceedings of the 2009 IEEE international conference on Multimedia and Expo
Semi-Supervised Learning

Semi-Supervised Learning
Leveraging active learning for relevance feedback using an information theoretic diversity measure

CIVR'06 Proceedings of the 5th international conference on Image and Video Retrieval
Relevance feedback: a power tool for interactive content-based image retrieval

IEEE Transactions on Circuits and Systems for Video Technology

RoboGene: an image retrieval system with multi-level log-based relevance feedback scheme

MMM'11 Proceedings of the 17th international conference on Advances in multimedia modeling - Volume Part II
Query difficulty guided image retrieval system

MMM'11 Proceedings of the 17th international conference on Advances in multimedia modeling - Volume Part II
ShotTagger: tag location for internet videos

Proceedings of the 1st ACM International Conference on Multimedia Retrieval
Locally regressive G-optimal design for image retrieval

Proceedings of the 1st ACM International Conference on Multimedia Retrieval
VisionGo: Towards video retrieval with joint exploration of human and computer

Information Sciences: an International Journal
Face image annotation and retrieval in impressive words using minimum bounding rectangles of face parts

KES'11 Proceedings of the 15th international conference on Knowledge-based and intelligent information and engineering systems - Volume Part IV
A pseudo relevance feedback based cross domain video concept detection

Proceedings of the Third International Conference on Internet Multimedia Computing and Service
Videoader: a video advertising system based on intelligent analysis of visual content

Proceedings of the Third International Conference on Internet Multimedia Computing and Service
Action retrieval with relevance feedback on YouTube videos

Proceedings of the Third International Conference on Internet Multimedia Computing and Service
Tag-based social image search with visual-text joint hypergraph learning

MM '11 Proceedings of the 19th ACM international conference on Multimedia
Integrating rich information for video recommendation with multi-task rank aggregation

MM '11 Proceedings of the 19th ACM international conference on Multimedia
Learning heterogeneous data for hierarchical web video classification

MM '11 Proceedings of the 19th ACM international conference on Multimedia
Intelligent photo clustering with user interaction and distance metric learning

Pattern Recognition Letters
k-Partite graph reinforcement and its application in multimedia information retrieval

Information Sciences: an International Journal
Assistive tagging: A survey of multimedia tagging with human-computer joint exploration

ACM Computing Surveys (CSUR)
Active learning for social image retrieval using Locally Regressive Optimal Design

Neurocomputing
In-video product annotation with web information mining

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
Attribute feedback

Proceedings of the 20th ACM international conference on Multimedia
Active learning based intervertebral disk classification combining shape and texture similarities

Neurocomputing
Active SVM-based relevance feedback using multiple classifiers ensemble and features reweighting

Engineering Applications of Artificial Intelligence
Measuring the Visual Complexities of Web Pages

ACM Transactions on the Web (TWEB)
Asymmetric propagation based batch mode active learning for image retrieval

Signal Processing
Multimedia encyclopedia construction by mining web knowledge

Signal Processing
Compressed domain based pornographic image recognition using multi-cost sensitive decision trees

Signal Processing
Literature survey of active learning in multimedia annotation and retrieval

Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service
A probabilistic model of active learning with multiple noisy oracles

Neurocomputing
Photo 4W: Mobile photo management on what, where, who and when

Neurocomputing
Advertising object in web videos

Neurocomputing
Certainty-based active learning for sampling imbalanced datasets

Neurocomputing
Active learning for human action retrieval using query pool selection

Neurocomputing
An image retrieval scheme with relevance feedback using feature reconstruction and SVM reclassification

Neurocomputing
Fuzzy deep belief networks for semi-supervised sentiment classification

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Active learning is a machine learning technique that selects the most informative samples for labeling and uses them as training data. It has been widely explored in multimedia research community for its capability of reducing human annotation effort. In this article, we provide a survey on the efforts of leveraging active learning in multimedia annotation and retrieval. We mainly focus on two application domains: image/video annotation and content-based image retrieval. We first briefly introduce the principle of active learning and then we analyze the sample selection criteria. We categorize the existing sample selection strategies used in multimedia annotation and retrieval into five criteria: risk reduction, uncertainty, diversity, density and relevance. We then introduce several classification models used in active learning-based multimedia annotation and retrieval, including semi-supervised learning, multilabel learning and multiple instance learning. We also provide a discussion on several future trends in this research direction. In particular, we discuss cost analysis of human annotation and large-scale interactive multimedia annotation.