Query by semantic example

Authors:
Nikhil Rasiwasia;Nuno Vasconcelos;Pedro J. Moreno
Affiliations:
Statistical Visual Computing Lab, University of California, San Diego;Statistical Visual Computing Lab, University of California, San Diego;Google, Inc., New York, NY
Venue:
CIVR'06 Proceedings of the 5th international conference on Image and Video Retrieval
Year:
2006

Citing 8
Cited 8

Unifying textual and visual cues for content-based image retrieval on the World Wide Web

Computer Vision and Image Understanding - Special issue on content-based access for image and video libraries
Content-Based Image Retrieval at the End of the Early Years

IEEE Transactions on Pattern Analysis and Machine Intelligence
Digital Libraries: Meeting Place for Low-Level And High-Level Vision

ACCV '95 Invited Session Papers from the Second Asian Conference on Computer Vision: Recent Developments in Computer Vision
Modeling annotated data

Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Matching words and pictures

The Journal of Machine Learning Research
Formulating Semantic Image Annotation as a Supervised Learning Problem

CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Multiple Bernoulli relevance models for image and video annotation

CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
Minimum probability of error image retrieval

IEEE Transactions on Signal Processing

Semantic concept-based query expansion and re-ranking for multimedia retrieval

Proceedings of the 15th international conference on Multimedia
Exploring multimedia in a keyword space

MM '08 Proceedings of the 16th ACM international conference on Multimedia
Image retrieval using query by contextual example

MIR '08 Proceedings of the 1st ACM international conference on Multimedia information retrieval
An exploratory study on joint analysis of visual classification in narrow domains and the discriminative power of tags

MS '08 Proceedings of the 2nd ACM workshop on Multimedia semantics
Semantic queries in databases: problems and challenges

Proceedings of the 18th ACM conference on Information and knowledge management
Cov-HGMEM: an improved hierarchical clustering algorithm

AIRS'08 Proceedings of the 4th Asia information retrieval conference on Information retrieval technology
A review on automatic image annotation techniques

Pattern Recognition
Semantic queries by example

Proceedings of the 16th International Conference on Extending Database Technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

A solution to the problem of image retrieval based on query-by-semantic-example (QBSE) is presented. QBSE extends the idea of query-by-example to the domain of semantic image representations. A semantic vocabulary is first defined, and a semantic retrieval system is trained to label each image with the posterior probability of appearance of each concept in the vocabulary. The resulting vector is interpreted as the projection of the image onto a semantic probability simplex, where a suitable similarity function is defined. Queries are specified by example images, which are projected onto the probability simplex. The database images whose projections on the simplex are closer to that of the query are declared its closest neighbors. Experimental evaluation indicates that 1) QBSE significantly outperforms the traditional query-by-visual-example paradigm when the concepts in the query image are known to the retrieval system, and 2) has equivalent performance even in the worst case scenario of queries composed by unknown concepts.