Automatic annotation of multimedia content by user clickthroughs: enhancing the performance of multimedia search engines

Authors:
Klimis Ntalianis;Anastasios Doulamis;Nicolas Tsapatsoulis;Nikolaos Doulamis
Affiliations:
Electrical and Computer Engineering Department, National Technical University of Athens, Zografou, Athens, Greece;Department of Production Engineering and Management, Technical Univeristy of Crete, Chania, Greece;Department of Communication and Internet Studies, Cyprus University of Technology, Limmasol, Cyprus;Electrical and Computer Engineering Department, National Technical University of Athens, Zografou, Athens, Greece
Venue:
MAMECTIS'08 Proceedings of the 10th WSEAS international conference on Mathematical methods, computational techniques and intelligent systems
Year:
2008

Citing 16
Cited 0

Automatic text processing

Automatic text processing
A fuzzy video content representation for video summarization and content-based retrieval

Signal Processing - Special issue on fuzzy logic in signal processing
Optimizing search engines using clickthrough data

Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
Indoor-Outdoor Image Classification

CAIVD '98 Proceedings of the 1998 International Workshop on Content-Based Access of Image and Video Databases (CAIVD '98)
Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach

IEEE Transactions on Pattern Analysis and Machine Intelligence
Boosting Image Retrieval

International Journal of Computer Vision - Special Issue on Content-Based Image Retrieval
Image Categorization by Learning and Reasoning with Regions

The Journal of Machine Learning Research
Formulating Semantic Image Annotation as a Supervised Learning Problem

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Region based image annotation through multiple-instance learning

Proceedings of the 13th annual ACM international conference on Multimedia
Learning the semantics of multimedia queries and concepts from a small number of examples

Proceedings of the 13th annual ACM international conference on Multimedia
Evaluation campaigns and TRECVid

MIR '06 Proceedings of the 8th ACM international workshop on Multimedia information retrieval
Real-Time Computerized Annotation of Pictures

IEEE Transactions on Pattern Analysis and Machine Intelligence
Toward Bridging the Annotation-Retrieval Gap in Image Search

IEEE MultiMedia
On supervision and statistical learning for semantic multimedia analysis

Journal of Visual Communication and Image Representation
Image classification for content-based indexing

IEEE Transactions on Image Processing
CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines

IEEE Transactions on Circuits and Systems for Video Technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

Content-based multimedia retrieval is a very hot research topic, applicable to several domains. Traditional feature vector based retrieval methods cannot provide semantically meaningful results. Additionally manual annotation is both time/money consuming and user-dependent. To address these problems in this paper we present an approach to automatically annotate multimedia files by incorporating clickthrough data of search engines. In particular the query-log of the search engine in connection with the log of links the users clicked on in the presented ranking, are analyzed in order to assign keywords to selected content. A query extension method is also proposed in order to agitate the pool of files and bring content with similar visual features to the surface. This is very important since users typically select only the first files of the ranking by clicking on them. The proposed method is feasible even for large sets of queries and features and theoretical results are verified in a controlled experiment, which shows that the method can effectively annotate multimedia files and significantly enhance the performance of multimedia search engines.