A critical investigation of recall and precision as measures of retrieval system performance
ACM Transactions on Information Systems (TOIS)
A re-examination of relevance: toward a dynamic, situational definition
Information Processing and Management: an International Journal
A relevance feedback mechanism for content-based image retrieval
Information Processing and Management: an International Journal
Content-Based Image Retrieval at the End of the Early Years
IEEE Transactions on Pattern Analysis and Machine Intelligence
Content-based query of image databases: inspirations from text retrieval
Pattern Recognition Letters - Selected papers from the 11th scandinavian conference on image analysis
Performance evaluation in content-based image retrieval: overview and proposals
Pattern Recognition Letters - Special issue on image/video indexing and retrieval
Support vector machine active learning for image retrieval
MULTIMEDIA '01 Proceedings of the ninth ACM international conference on Multimedia
Learning and inferring a semantic space from user's relevance feedback for image retrieval
Proceedings of the tenth ACM international conference on Multimedia
Benchmarking for Content-Based Visual Information Search
VISUAL '00 Proceedings of the 4th International Conference on Advances in Visual Information Systems
The Truth about Corel - Evaluation in Image Retrieval
CIVR '02 Proceedings of the International Conference on Image and Video Retrieval
The Philosophy of Information Retrieval Evaluation
CLEF '01 Revised Papers from the Second Workshop of the Cross-Language Evaluation Forum on Evaluation of Cross-Language Information Retrieval Systems
Multi-class relevance feedback content-based image retrieval
Computer Vision and Image Understanding
International Journal of Computer Vision - Special Issue on Content-Based Image Retrieval
Mean version space: a new active learning method for content-based image retrieval
Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval
Long-term learning of semantic grouping from relevance-feedback
Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval
A novel log-based relevance feedback technique in content-based image retrieval
Proceedings of the 12th annual ACM international conference on Multimedia
Content-based sub-image retrieval using relevance feedback
Proceedings of the 2nd ACM international workshop on Multimedia databases
IEEE Transactions on Pattern Analysis and Machine Intelligence
Semantic Kernel Updating for Content-Based Image Retrieval
ISMSE '04 Proceedings of the IEEE Sixth International Symposium on Multimedia Software Engineering
Kernel Vector Approximation Files for Relevance Feedback Retrieval in Large Image Databases
Multimedia Tools and Applications
Semantic manifold learning for image retrieval
Proceedings of the 13th annual ACM international conference on Multimedia
Content-based image retrieval: approaches and trends of the new age
Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval
Content-based multimedia information retrieval: State of the art and challenges
ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
Large-Scale Concept Ontology for Multimedia
IEEE MultiMedia
Benchmarking image and video retrieval: an overview
MIR '06 Proceedings of the 8th ACM international workshop on Multimedia information retrieval
Usage-oriented multimedia information retrieval technological evaluation
MIR '06 Proceedings of the 8th ACM international workshop on Multimedia information retrieval
Human performance measures for video retrieval
MIR '06 Proceedings of the 8th ACM international workshop on Multimedia information retrieval
Evaluation campaigns and TRECVid
MIR '06 Proceedings of the 8th ACM international workshop on Multimedia information retrieval
The challenge problem for automated detection of 101 semantic concepts in multimedia
MULTIMEDIA '06 Proceedings of the 14th annual ACM international conference on Multimedia
Multimedia research challenges for industry
CIVR'05 Proceedings of the 4th international conference on Image and Video Retrieval
Aspect-based relevance learning for image retrieval
CIVR'05 Proceedings of the 4th international conference on Image and Video Retrieval
The CLEF 2004 cross-language image retrieval track
CLEF'04 Proceedings of the 5th conference on Cross-Language Evaluation Forum: multilingual Information Access for Text, Speech and Images
Image searching and browsing by active aspect-based relevance learning
CIVR'06 Proceedings of the 5th international conference on Image and Video Retrieval
IEEE Transactions on Image Processing
Relevance feedback: a power tool for interactive content-based image retrieval
IEEE Transactions on Circuits and Systems for Video Technology
The MIR flickr retrieval evaluation
MIR '08 Proceedings of the 1st ACM international conference on Multimedia information retrieval
New trends and ideas in visual concept detection: the MIR flickr retrieval evaluation initiative
Proceedings of the international conference on Multimedia information retrieval
Minimum explanation complexity for MOD based visual concept detection
Proceedings of the international conference on Multimedia information retrieval
Relevance feedback strategies for artistic image collections tagging
Proceedings of the 1st ACM International Conference on Multimedia Retrieval
Feature space warping relevance feedback with transductive learning
ACIVS'11 Proceedings of the 13th international conference on Advanced concepts for intelligent vision systems
Active SVM-based relevance feedback using multiple classifiers ensemble and features reweighting
Engineering Applications of Artificial Intelligence
| Hi-index | 0.00 |
In this paper we review the evaluation of relevance feedback methods for content-based image retrieval systems. We start out by presenting an overview of current common practice, and argue that the evaluation of relevance feedback methods differs from evaluating CBIR systems as a whole. Specifically, we identify the challenging issues that are particular to the evaluation of retrieval employing relevance feedback. Next, we propose three guidelines to move toward more effective evaluation benchmarks. We focus particularly on assessing feedback methods more directly in terms of their goal of identifying the relevant target class with a small number of samples, and show how to compensate for query targets of varying difficulty by measuring efficiency at generalization.