Attribute-augmented semantic hierarchy: towards bridging semantic gap and intention gap in image retrieval

Authors:
Hanwang Zhang;Zheng-Jun Zha;Yang Yang;Shuicheng Yan;Yue Gao;Tat-Seng Chua
Affiliations:
School of Computing, National University of Singapore, Singapore, Singapore;Institute of Intelligent Machines, Chinese Academy of Sciences, He Fei, China;School of Computing, National University of Singapore, Singapore, Singapore;Electrical Computer Engineering, Chinese Academy of Sciences, Singapore, Singapore;School of Computing, National University of Singapore, Singapore, Singapore;School of Computing, National University of Singapore, Singapore, Singapore
Venue:
Proceedings of the 21st ACM international conference on Multimedia
Year:
2013

Citing 26
Cited 0

VisualSEEk: a fully automated content-based image query system

MULTIMEDIA '96 Proceedings of the fourth ACM international conference on Multimedia
Content-Based Image Retrieval at the End of the Early Years

IEEE Transactions on Pattern Analysis and Machine Intelligence
Support vector machine active learning for image retrieval

MULTIMEDIA '01 Proceedings of the ninth ACM international conference on Multimedia
Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons

International Journal of Computer Vision
On image auto-annotation with latent space models

MULTIMEDIA '03 Proceedings of the eleventh ACM international conference on Multimedia
Distinctive Image Features from Scale-Invariant Keypoints

International Journal of Computer Vision
Content-based multimedia information retrieval: State of the art and challenges

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
Asymmetric Bagging and Random Subspace for Support Vector Machines-Based Relevance Feedback in Image Retrieval

IEEE Transactions on Pattern Analysis and Machine Intelligence
Large-Scale Concept Ontology for Multimedia

IEEE MultiMedia
Image retrieval: Ideas, influences, and trends of the new age

ACM Computing Surveys (CSUR)
Concept-Based Video Retrieval

Foundations and Trends in Information Retrieval
Visual query suggestion

MM '09 Proceedings of the 17th ACM international conference on Multimedia
Maximum margin clustering made practical

IEEE Transactions on Neural Networks
The state of the art in image and video retrieval

CIVR'03 Proceedings of the 2nd international conference on Image and video retrieval
Theory and Applications of Ontology: Computer Applications

Theory and Applications of Ontology: Computer Applications
Interactively building a discriminative vocabulary of nameable attributes

CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
Visual and semantic similarity in ImageNet

CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
Combining attributes and Fisher vectors for efficient image retrieval

CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
Adding Semantics to Detectors for Video Retrieval

IEEE Transactions on Multimedia
Integrating Concept Ontology and Multitask Learning to Achieve More Effective Classifier Training for Multilevel Image Annotation

IEEE Transactions on Image Processing
Relevance feedback: a power tool for interactive content-based image retrieval

IEEE Transactions on Circuits and Systems for Video Technology
WhittleSearch: Image search with relative attribute feedback

CVPR '12 Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Attribute feedback

Proceedings of the 20th ACM international conference on Multimedia
Intent and its discontents: the user at the wheel of the online video search engine

Proceedings of the 20th ACM international conference on Multimedia
Attribute learning in large-scale datasets

ECCV'10 Proceedings of the 11th European conference on Trends and Topics in Computer Vision - Volume Part I
Complex Event Detection via Multi-source Video Attributes

CVPR '13 Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a novel Attribute-augmented Semantic Hierarchy (A2 SH) and demonstrates its effectiveness in bridging both the semantic and intention gaps in Content-based Image Retrieval (CBIR). A2 SH organizes the semantic concepts into multiple semantic levels and augments each concept with a set of related attributes, which describe the multiple facets of the concept and act as the intermediate bridge connecting the concept and low-level visual content. A hierarchical semantic similarity function is learnt to characterize the semantic similarities among images for retrieval. To better capture user search intent, a hybrid feedback mechanism is developed, which collects hybrid feedbacks on attributes and images. These feedbacks are then used to refine the search results based on A2 SH. We develop a content-based image retrieval system based on the proposed A2 SH. We conduct extensive experiments on a large-scale data set of over one million Web images. Experimental results show that the proposed A2 SH can characterize the semantic affinities among images accurately and can shape user search intent precisely and quickly, leading to more accurate search results as compared to state-of-the-art CBIR solutions.