Content-based image retrieval using a combination of visual features and eye tracking data

Authors:
Zhen Liang;Hong Fu;Yun Zhang;Zheru Chi;Dagan Feng
Affiliations:
The Hong Kong Polytechnic University, Hong Kong, China;The Hong Kong Polytechnic University, Hong Kong, China;The Hong Kong Polytechnic University, Hong Kong, China;The Hong Kong Polytechnic University, Hong Kong, China;The Hong Kong Polytechnic University, Hong Kong, China and The University of Sydney, Sydney, Australia
Venue:
Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications
Year:
2010

Citing 8
Cited 2

Content-Based Image Retrieval at the End of the Early Years

IEEE Transactions on Pattern Analysis and Machine Intelligence
Unsupervised Segmentation of Color-Texture Regions in Images and Video

IEEE Transactions on Pattern Analysis and Machine Intelligence
Image classification using hybrid neural networks

Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Image similarity search with compact data structures

Proceedings of the thirteenth ACM international conference on Information and knowledge management
Using visual attention to extract regions of interest in the context of image retrieval

Proceedings of the 44th annual Southeast regional conference
Attention-driven image interpretation with application to image retrieval

Pattern Recognition
Is bottom-up attention useful for object recognition?

CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
Perceptually uniform color spaces for color texture analysis: an empirical evaluation

IEEE Transactions on Image Processing

An eye-tracking-based approach to facilitate interactive video search

Proceedings of the 1st ACM International Conference on Multimedia Retrieval
The colour and texture - a novel image retrieval technology based on human vision

International Journal of Innovative Computing and Applications

Quantified Score

Hi-index	0.00

Visualization

Abstract

Image retrieval technology has been developed for more than twenty years. However, the current image retrieval techniques cannot achieve a satisfactory recall and precision. To improve the effectiveness and efficiency of an image retrieval system, a novel content-based image retrieval method with a combination of image segmentation and eye tracking data is proposed in this paper. In the method, eye tracking data is collected by a non-intrusive table mounted eye tracker at a sampling rate of 120 Hz, and the corresponding fixation data is used to locate the human's Regions of Interest (hROIs) on the segmentation result from the JSEG algorithm. The hROIs are treated as important informative segments/objects and used in the image matching. In addition, the relative gaze duration of each hROI is used to weigh the similarity measure for image retrieval. The similarity measure proposed in this paper is based on a retrieval strategy emphasizing the most important regions. Experiments on 7346 Hemera color images annotated manually show that the retrieval results from our proposed approach compare favorably with conventional content-based image retrieval methods, especially when the important regions are difficult to be located based on visual features.