Viewing morphology as an inference process
SIGIR '93 Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval
A study of smoothing methods for language models applied to Ad Hoc information retrieval
Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval
SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Query association surrogates for Web search: Research Articles
Journal of the American Society for Information Science and Technology
On ranking the effectiveness of searches
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Query performance prediction in web search environments
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Bolasso: model consistent Lasso estimation through the bootstrap
Proceedings of the 25th international conference on Machine learning
Extended gloss overlaps as a measure of semantic relatedness
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Effective pre-retrieval query performance prediction using similarity and variability evidence
ECIR'08 Proceedings of the IR research, 30th European conference on Advances in information retrieval
Using coherence-based measures to predict query difficulty
ECIR'08 Proceedings of the IR research, 30th European conference on Advances in information retrieval
A Belief Model of Query Difficulty That Uses Subjective Logic
ICTIR '09 Proceedings of the 2nd International Conference on Theory of Information Retrieval: Advances in Information Retrieval Theory
Evaluation of query performance prediction methods by range
SPIRE'10 Proceedings of the 17th international conference on String processing and information retrieval
A large-scale system evaluation on component-level
ECIR'11 Proceedings of the 33rd European conference on Advances in information retrieval
Time-based query performance predictors
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
A unified framework for post-retrieval query-performance prediction
ICTIR'11 Proceedings of the Third international conference on Advances in information retrieval theory
A performance prediction approach to enhance collaborative filtering performance
ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
Oracle in Image Search: A Content-Based Approach to Performance Prediction
ACM Transactions on Information Systems (TOIS)
Query performance prediction for IR
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Back to the roots: a probabilistic framework for query-performance prediction
Proceedings of the 21st ACM international conference on Information and knowledge management
Query-performance prediction and cluster ranking: two sides of the same coin
Proceedings of the 21st ACM international conference on Information and knowledge management
Estimating query difficulty for news prediction retrieval
Proceedings of the 21st ACM international conference on Information and knowledge management
Correlating medical-dependent query features with image retrieval models using association rules
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
| Hi-index | 0.01 |
In this paper, we examine a number of newly applied methods for combining pre-retrieval query performance predictors in order to obtain a better prediction of the query's performance. However, in order to adequately and appropriately compare such techniques, we critically examine the current evaluation methodology and show how using linear correlation coefficients (i) do not provide an intuitive measure indicative of a method's quality, (ii) can provide a misleading indication of performance, and (iii) overstate the performance of combined methods. To address this, we extend the current evaluation methodology to include cross validation, report a more intuitive and descriptive statistic, and apply statistical testing to determine significant differences. During the course of a comprehensive empirical study over several TREC collections, we evaluate nineteen pre-retrieval predictors and three combination methods.