Cross-media retrieval using query dependent search methods

  • Authors:
  • Yi Yang;Fei Wu;Dong Xu;Yueting Zhuang;Liang-Tien Chia

  • Affiliations:
  • College of Computer Science, Zhejiang University, PR China;Institute of Artificial Intelligence, College of Computer Science, Zhejiang University, Hangzhou 310027, PR China;School of Computer Engineering, Nanyang Technological University, Singapore;College of Computer Science, Zhejiang University, PR China;School of Computer Engineering, Nanyang Technological University, Singapore

  • Venue:
  • Pattern Recognition
  • Year:
  • 2010

Quantified Score

Hi-index 0.01

Visualization

Abstract

The content-based cross-media retrieval is a new type of multimedia retrieval in which the media types of query examples and the returned results can be different. In order to learn the semantic correlations among multimedia objects of different modalities, the heterogeneous multimedia objects are analyzed in the form of multimedia document (MMD), which is a set of multimedia objects that are of different media types but carry the same semantics. We first construct an MMD semi-semantic graph (MMDSSG) by jointly analyzing the heterogeneous multimedia data. After that, cross-media indexing space (CMIS) is constructed. For each query, the optimal dimension of CMIS is automatically determined and the cross-media retrieval is performed on a per-query basis. By doing this, the most appropriate retrieval approach for each query is selected, i.e. different search methods are used for different queries. The query dependent search methods make cross-media retrieval performance not only accurate but also stable. We also propose different learning methods of relevance feedback (RF) to improve the performance. Experiment is encouraging and validates the proposed methods.