Using evidences based on natural language to drive the process of fusing multimodal sources

Authors:
Sergio Navarro;Fernando Llopis;Rafael Muñoz
Affiliations:
Natural Language Processing and Information Systems Group, University of Alicante, Spain;Natural Language Processing and Information Systems Group, University of Alicante, Spain;Natural Language Processing and Information Systems Group, University of Alicante, Spain
Venue:
NLDB'09 Proceedings of the 14th international conference on Applications of Natural Language to Information Systems
Year:
2009

Citing 7
Cited 1

Improving the effectiveness of information retrieval with local context analysis

ACM Transactions on Information Systems (TOIS)
Probabilistic models of information retrieval based on measuring the divergence from randomness

ACM Transactions on Information Systems (TOIS)
Improving the Effectiveness of Local Context Analysis Based on Semantic Similarity

ICCIT '07 Proceedings of the 2007 International Conference on Convergence Information Technology
Features for image retrieval: an experimental comparison

Information Retrieval
Overview of the ImageCLEFphoto 2007 Photographic Retrieval Task

Advances in Multilingual and Multimodal Information Retrieval
Overview of the WikipediaMM task at ImageCLEF 2008

CLEF'08 Proceedings of the 9th Cross-language evaluation forum conference on Evaluating systems for multilingual and multimodal information access
Overview of the ImageCLEF 2006 photographic retrieval and object annotation tasks

CLEF'06 Proceedings of the 7th international conference on Cross-Language Evaluation Forum: evaluation of multilingual and multi-modal information retrieval

Diversity promotion: is reordering top-ranked documents sufficient?

CLEF'09 Proceedings of the 10th international conference on Cross-language evaluation forum: multimedia experiments

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper focuses on the proposal and evaluation of two multimodal fusion techniques in the field of Visual Information Retrieval (VIR). These proposals are based on two widely used fusion strategies in the VIR area, the multimodal blind relevance feedback and the multimodal re-ranking strategy. Unlike the existent techniques, our alternative proposals are guided by the evidence found in the natural language annotations related to the images. The results achieved by our runs in two different ImageCLEF tasks, 3rd place in the Wikipedia task [1] and 4th place within all the automatic runs in the photo task [2], jointly with the results obtained in later experiments presented in this paper show us that the use of conceptual information associated with an image can improve significantly the performance of the original multimodal fusion techniques used.