Two axioms for evaluation measures in information retrieval
Proc. of the third joint BCS and ACM symposium on Research and development in information retrieval
Term-weighting approaches in automatic text retrieval
Information Processing and Management: an International Journal
A utility-theoretic analysis of expected search length
SIGIR '88 Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval
Recent trends in automatic information retrieval
Proceedings of the 9th annual international ACM SIGIR conference on Research and development in information retrieval
Computer Evaluation of Indexing and Text Processing
Journal of the ACM (JACM)
Precision Weighting—An Effective Automatic Indexing Method
Journal of the ACM (JACM)
Information Retrieval
Introduction to Modern Information Retrieval
Introduction to Modern Information Retrieval
Measurement-theoretical investigation of the MZ-metric
SIGIR '80 Proceedings of the 3rd annual ACM conference on Research and development in information retrieval
Implementation of the SMART Information Retrieval System
Implementation of the SMART Information Retrieval System
The SMART Retrieval System—Experiments in Automatic Document Processing
The SMART Retrieval System—Experiments in Automatic Document Processing
Phonetic string matching: lessons from information retrieval
SIGIR '96 Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval
How reliable are the results of large-scale information retrieval experiments?
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
Content-Based Image Retrieval in Astronomy
Information Retrieval
Association and Content-Based Retrieval
IEEE Transactions on Knowledge and Data Engineering
Image Access and Data Mining: An Approach
PKDD '00 Proceedings of the 4th European Conference on Principles of Data Mining and Knowledge Discovery
Dynamic Models of Expert Groups to Recommend Web Documents
ECDL '01 Proceedings of the 5th European Conference on Research and Advanced Technology for Digital Libraries
An Automated ILP Server in the Field of Bioinformatics
ILP '01 Proceedings of the 11th International Conference on Inductive Logic Programming
Current Status of the Evaluation of Information Retrieval
Journal of Medical Systems
Variable-length intervals in homology search
APBC '04 Proceedings of the second conference on Asia-Pacific bioinformatics - Volume 29
Report on the INEX 2003 workshop
ACM SIGIR Forum
A report on the first year of the INitiative for the evaluation of XML retrieval (INEX'02)
Journal of the American Society for Information Science and Technology
The overlap problem in content-oriented XML retrieval evaluation
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Interpolation models for spatiotemporal association mining
Fundamenta Informaticae - Special issue on the 9th international conference on rough sets, fuzzy sets, data mining and granular computing (RSFDGrC 2003)
A hybrid system for affine-invariant trajectory retrieval
Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval
IEEE Transactions on Pattern Analysis and Machine Intelligence
The maximum entropy method for analyzing retrieval measures
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
IEEE Transactions on Knowledge and Data Engineering
A geometric interpretation and analysis of R-precision
Proceedings of the 14th ACM international conference on Information and knowledge management
On the development of name search techniques for Arabic
Journal of the American Society for Information Science and Technology
The relationship between Precision-Recall and ROC curves
ICML '06 Proceedings of the 23rd international conference on Machine learning
Evaluation in (XML) information retrieval: expected precision-recall with user modelling (EPRUM)
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Investigating the exhaustivity dimension in content-oriented XML element retrieval evaluation
CIKM '06 Proceedings of the 15th ACM international conference on Information and knowledge management
Precision recall with user modeling (PRUM): Application to structured information retrieval
ACM Transactions on Information Systems (TOIS)
Red Opal: product-feature scoring from reviews
Proceedings of the 8th ACM conference on Electronic commerce
Evaluating XML retrieval effectiveness at INEX
ACM SIGIR Forum
A similarity-based method for retrieving documents from the SCI/SSCI database
Journal of Information Science
Search Strategies in Shopping Engines: An Experimental Investigation
International Journal of Electronic Commerce
SPICE: A New Framework for Data Mining based on Probability Logic and Formal Concept Analysis
Fundamenta Informaticae - Special issue ISMIS'05
Tracking reliability and helpfulness in agent interactions
Web Intelligence and Agent Systems
Performance evaluation of relevance feedback methods
CIVR '08 Proceedings of the 2008 international conference on Content-based image and video retrieval
Estimating average precision when judgments are incomplete
Knowledge and Information Systems
Rank-biased precision for measurement of retrieval effectiveness
ACM Transactions on Information Systems (TOIS)
Structural relevance: a common basis for the evaluation of structured document retrieval
Proceedings of the 17th ACM conference on Information and knowledge management
Nonparametric estimation of the precision-recall curve
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Information Extraction System Based on Hidden Markov Model
ISNN '09 Proceedings of the 6th International Symposium on Neural Networks on Advances in Neural Networks
Non-contiguous word sequences for information retrieval
MWE '04 Proceedings of the Workshop on Multiword Expressions: Integrating Processing
Soft-supervised learning for text classification
EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
New Regularized Algorithms for Transductive Learning
ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II
Contradiction and Correlation for Camera Overlap Estimation
AVSS '09 Proceedings of the 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance
Episode-constrained cross-validation in video concept retrieval
IEEE Transactions on Multimedia
Finding variants of out-of-vocabulary words in Arabic
Semitic '07 Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources
Ranking web documents with dynamic evaluation by expert groups
CAiSE'03 Proceedings of the 15th international conference on Advanced information systems engineering
A robust caption detecting algorithm on MPEG compressed video
MCAM'07 Proceedings of the 2007 international conference on Multimedia content analysis and mining
Interpolation techniques for geo-spatial association rule mining
RSFDGrC'03 Proceedings of the 9th international conference on Rough sets, fuzzy sets, data mining, and granular computing
Computing information retrieval performance measures efficiently in the presence of tied scores
ECIR'08 Proceedings of the IR research, 30th European conference on Advances in information retrieval
Tie-breaking bias: effect of an uncontrolled parameter on information retrieval evaluation
CLEF'10 Proceedings of the 2010 international conference on Multilingual and multimodal information access evaluation: cross-language evaluation forum
Looking for "good" recommendations: a comparative evaluation of recommender systems
INTERACT'11 Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part III
What Makes a Phone a Business Phone - Querying Concepts in Product Data
WI-IAT '11 Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01
Semi-Supervised Learning with Measure Propagation
The Journal of Machine Learning Research
Performance evaluation of image segmentation
ICIAR'06 Proceedings of the Third international conference on Image Analysis and Recognition - Volume Part I
INEX'05 Proceedings of the 4th international conference on Initiative for the Evaluation of XML Retrieval
On effectiveness measures and relevance functions in ranking INEX systems
AIRS'05 Proceedings of the Second Asia conference on Asia Information Retrieval Technology
Correlation filters for facial recognition login access control
PCM'04 Proceedings of the 5th Pacific Rim conference on Advances in Multimedia Information Processing - Volume Part I
A comprehensive OWA-based framework for result merging in metasearch
RSFDGrC'05 Proceedings of the 10th international conference on Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing - Volume Part II
Computer Vision and Image Understanding
Complete performance graphs in probabilistic information retrieval
PCM'04 Proceedings of the 5th Pacific Rim Conference on Advances in Multimedia Information Processing - Volume Part II
Reliability tests for the XCG and inex-2002 metrics
INEX'04 Proceedings of the Third international conference on Initiative for the Evaluation of XML Retrieval
ACM Transactions on Interactive Intelligent Systems (TiiS)
Hybrid Matching Algorithm for Personal Names
Journal of Data and Information Quality (JDIQ)
SPICE: A New Framework for Data Mining based on Probability Logic and Formal Concept Analysis
Fundamenta Informaticae - Special issue ISMIS'05
Interactive information retrieval algorithm for wikipedia articles
IDEAL'12 Proceedings of the 13th international conference on Intelligent Data Engineering and Automated Learning
Interpolation Models for Spatiotemporal Association Mining
Fundamenta Informaticae - The 9th International Conference on Rough Sets, Fuzzy Sets, Data Mining and Granular Conputing (RSFDGrC 2003)
Cloning mechanisms to improve agent performances
Journal of Network and Computer Applications
Design and Analysis of Classifier Learning Experiments in Bioinformatics: Survey and Case Studies
IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
K-local hyperplane distance nearest neighbor classifier oriented local discriminant analysis
Information Sciences: an International Journal
A taxonomy of privacy-preserving record linkage techniques
Information Systems
An efficient two-party protocol for approximate matching in private record linkage
AusDM '11 Proceedings of the Ninth Australasian Data Mining Conference - Volume 121
Information quality measurement of medical encoding support based on usability
Computer Methods and Programs in Biomedicine
Estimation of a Priori Decision Threshold for Collocations Extraction: An Empirical Study
International Journal of Information Technology and Web Engineering
Querying concepts in product data by means of query expansion
Web Intelligence and Agent Systems
| Hi-index | 0.00 |
Recall and precision are often used to evaluate the effectiveness of information retrieval systems. They are easy to define if there is a single query and if the retrieval result generated for the query is a linear ordering. However, when the retrieval results are weakly ordered, in the sense that several documents have an identical retrieval status value with respect to a query, some probabilistic notion of precision has to be introduced. Relevance probability, expected precision, and so forth, are some alternatives mentioned in the literature for this purpose. Furthermore, when many queries are to be evaluated and the retrieval results averaged over these queries, some method of interpolation of precision values at certain preselected recall levels is needed. The currently popular approaches for handling both a weak ordering and interpolation are found to be inconsistent, and the results obtained are not easy to interpret. Moreover, in cases where some alternatives are available, no comparative analysis that would facilitate the selection of a particular strategy has been provided. In this paper, we systematically investigate the various problems and issues associated with the use of recall and precision as measures of retrieval system performance. Our motivation is to provide a comparative analysis of methods available for defining precision in a probabilistic sense and to promote a better understanding of the various issues involved in retrieval performance evaluation.