VisualSEEk: a fully automated content-based image query system
MULTIMEDIA '96 Proceedings of the fourth ACM international conference on Multimedia
Performance evaluation in content-based image retrieval: overview and proposals
Pattern Recognition Letters - Special issue on image/video indexing and retrieval
Alternating feature spaces in relevance feedback
MULTIMEDIA '01 Proceedings of the 2001 ACM workshops on Multimedia: multimedia information retrieval
A novel region-based image retrieval method using relevance feedback
MULTIMEDIA '01 Proceedings of the 2001 ACM workshops on Multimedia: multimedia information retrieval
Support vector machine pairwise classifiers with error reduction for image classification
MULTIMEDIA '01 Proceedings of the 2001 ACM workshops on Multimedia: multimedia information retrieval
A statistical correlation model for image retrieval
MULTIMEDIA '01 Proceedings of the 2001 ACM workshops on Multimedia: multimedia information retrieval
SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries
IEEE Transactions on Pattern Analysis and Machine Intelligence
Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying
IEEE Transactions on Pattern Analysis and Machine Intelligence
Supporting Ranked Boolean Similarity Queries in MARS
IEEE Transactions on Knowledge and Data Engineering
Learning Over Multiple Temporal Scales in Image Databases
ECCV '00 Proceedings of the 6th European Conference on Computer Vision-Part I
A Rich Get Richer Strategy for Content-Based Image Retrieval
VISUAL '00 Proceedings of the 4th International Conference on Advances in Visual Information Systems
Efficient Query Refinement in Multimedia Databases
ICDE '00 Proceedings of the 16th International Conference on Data Engineering
The SMART Retrieval System—Experiments in Automatic Document Processing
The SMART Retrieval System—Experiments in Automatic Document Processing
Challenges of Image and Video Retrieval
CIVR '02 Proceedings of the International Conference on Image and Video Retrieval
Evaluation of key frame-based retrieval techniques for video
Computer Vision and Image Understanding - Special isssue on video retrieval and summarization
IEEE Transactions on Pattern Analysis and Machine Intelligence
Document Ranking by Layout Relevance
ICDAR '05 Proceedings of the Eighth International Conference on Document Analysis and Recognition
Ontological inference for image and video analysis
Machine Vision and Applications
Content-based multimedia information retrieval: State of the art and challenges
ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
Qualitative evaluation of automatic assignment of keywords to images
Information Processing and Management: an International Journal - Special issue: Formal methods for information retrieval
Benchmarking image and video retrieval: an overview
MIR '06 Proceedings of the 8th ACM international workshop on Multimedia information retrieval
A survey of content-based image retrieval with high-level semantics
Pattern Recognition
Modeling Semantic Aspects for Cross-Media Image Indexing
IEEE Transactions on Pattern Analysis and Machine Intelligence
Image retrieval: Ideas, influences, and trends of the new age
ACM Computing Surveys (CSUR)
Features for image retrieval: an experimental comparison
Information Retrieval
Towards automatic feature vector optimization for multimedia applications
Proceedings of the 2008 ACM symposium on Applied computing
Performance evaluation of relevance feedback methods
CIVR '08 Proceedings of the 2008 international conference on Content-based image and video retrieval
CIVR '08 Proceedings of the 2008 international conference on Content-based image and video retrieval
A knowledge based approach for a fast image retrieval system
Image and Vision Computing
A survey of methods for image annotation
Journal of Visual Languages and Computing
The MIR flickr retrieval evaluation
MIR '08 Proceedings of the 1st ACM international conference on Multimedia information retrieval
Semantic object classes in video: A high-definition ground truth database
Pattern Recognition Letters
Appearance based retrieval for tracked objects in surveillance videos
Proceedings of the ACM International Conference on Image and Video Retrieval
Shape reasoning on mis-segmented and mis-labeled objects using approximated Fisher criterion
Computers and Graphics
Qualitative evaluation of automatic assignment of keywords to images
Information Processing and Management: an International Journal - Special issue: Formal methods for information retrieval
Multimodal medical image retrieval: image categorization to improve search precision
Proceedings of the international conference on Multimedia information retrieval
Proceedings of the international conference on Multimedia information retrieval
The segmented and annotated IAPR TC-12 benchmark
Computer Vision and Image Understanding
Relevance feedback for content-based image retrieval: what can three mouse clicks achieve?
ECIR'03 Proceedings of the 25th European conference on IR research
Shape feature matching for trademark image retrieval
CIVR'03 Proceedings of the 2nd international conference on Image and video retrieval
Performance comparison of different similarity models for CBIR with relevance feedback
CIVR'03 Proceedings of the 2nd international conference on Image and video retrieval
Where are focused places of a photo?
VISUAL'07 Proceedings of the 9th international conference on Advances in visual information systems
A query language combining object features and semantic events for surveillance video retrieval
MMM'08 Proceedings of the 14th international conference on Advances in multimedia modeling
Semantic discriminative projections for image retrieval
AIRS'08 Proceedings of the 4th Asia information retrieval conference on Information retrieval technology
A fuzzy logic based approach to feedback reinforcement in image retrieval
ICIC'09 Proceedings of the 5th international conference on Emerging intelligent computing technology and applications
CLEF'08 Proceedings of the 9th Cross-language evaluation forum conference on Evaluating systems for multilingual and multimodal information access
IPSILON: incremental parsing for semantic indexing of latent concepts
IEEE Transactions on Image Processing
Unified tag analysis with multi-edge graph
Proceedings of the international conference on Multimedia
Semantic approach to image retrieval using statistical models based on a lexical ontology
KES'10 Proceedings of the 14th international conference on Knowledge-based and intelligent information and engineering systems: Part IV
Semantic analysis and retrieval in personal and social photo collections
Multimedia Tools and Applications
Approximation of linear discriminant analysis for word dependent visual features selection
ACIVS'05 Proceedings of the 7th international conference on Advanced Concepts for Intelligent Vision Systems
Automated image annotation using global features and robust nonparametric density estimation
CIVR'05 Proceedings of the 4th international conference on Image and Video Retrieval
Complete performance graphs in probabilistic information retrieval
PCM'04 Proceedings of the 5th Pacific Rim Conference on Advances in Multimedia Information Processing - Volume Part II
How to visually retrieve images from the st. andrews collection using GIFT
CLEF'04 Proceedings of the 5th conference on Cross-Language Evaluation Forum: multilingual Information Access for Text, Speech and Images
A novel framework for robust annotation and retrieval in video sequences
CIVR'06 Proceedings of the 5th international conference on Image and Video Retrieval
Toward consistent evaluation of relevance feedback approaches in multimedia retrieval
AMR'05 Proceedings of the Third international conference on Adaptive Multimedia Retrieval: user, context, and feedback
On the consistency and features of image similarity
Proceedings of the 4th Information Interaction in Context Symposium
Apples to oranges: evaluating image annotations from natural language processing systems
NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
A benchmark for content-based retrieval in bivariate data collections
TPDL'12 Proceedings of the Second international conference on Theory and Practice of Digital Libraries
Hi-index | 0.00 |
To demonstrate the performance of content-based image retrieval systems (CBIRSs), there is not yet any standard data set that is widely used. The only dataset used by a large number of research groups are the Corel Photo CDs. There are more than 800 of those CDs, each containing 100 pictures roughly similar in theme. Unfortunately, basically every evaluation is done on a different subset of the image sets thus making comparison impossible.In this article, we compare different ways of evaluating the performance using a subset of the Corel images with the same CBIRSan d the same set of evaluation measures. The aim is to show how easy it is to get differing results, even when using the same image collection, the same CBIRS and the same performance measures. This pinpoints the fact that we need a standard database of images with a query set and corresponding relevance judgments (RJs) to really compare systems.The techniques used in this article to "enhance" the apparent performance of a CBIRSa re commonly used, sometimes described, sometimes not. They all have a justification and seem to change the performance of a CBIRS but they do actually not. With a larger subset of images it is of course much easier to generate even bigger differences in performance. The goal of this article is not to be a guide of how to make the "apparent" performance of systems look good, but rather to make readers aware of CBIRS evaluations and the importance of standardized image databases, queries and RJ.