Elements of information theory
Elements of information theory
Ranking retrieval systems without relevance judgments
Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval
Evaluation by highly relevant documents
Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval
Evaluating strategies for similarity search on the web
Proceedings of the 11th international conference on World Wide Web
SODA '03 Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms
Methods for ranking information retrieval systems without relevance judgments
Proceedings of the 2003 ACM symposium on Applied computing
A unified model for metasearch, pooling, and system evaluation
CIKM '03 Proceedings of the twelfth international conference on Information and knowledge management
Retrieval evaluation with incomplete information
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Forming test collections with no system pooling
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Proceedings of the 14th ACM international conference on Information and knowledge management
Estimating average precision with incomplete and imperfect judgments
CIKM '06 Proceedings of the 15th ACM international conference on Information and knowledge management
On rank correlation in information retrieval evaluation
ACM SIGIR Forum
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
CARES: a ranking-oriented CADAL recommender system
Proceedings of the 9th ACM/IEEE-CS joint conference on Digital libraries
On rank correlation and the distance between rankings
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Nonlinear static-rank computation
Proceedings of the 18th ACM conference on Information and knowledge management
Weighted Rank Correlation in Information Retrieval Evaluation
AIRS '09 Proceedings of the 5th Asia Information Retrieval Symposium on Information Retrieval Technology
Studies on intrinsic summary evaluation
International Journal of Artificial Intelligence and Soft Computing
Generalized distances between rankings
Proceedings of the 19th international conference on World wide web
Visualizing differences in web search algorithms using the expected weighted hoeffding distance
Proceedings of the 19th international conference on World wide web
Do user preferences and evaluation measures line up?
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
A similarity measure for indefinite rankings
ACM Transactions on Information Systems (TOIS)
Score aggregation techniques in retrieval experimentation
ADC '09 Proceedings of the Twentieth Australasian Conference on Australasian Database - Volume 92
Web search solved?: all result rankings the same?
CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
ICDM'10 Proceedings of the 10th industrial conference on Advances in data mining: applications and theoretical aspects
Evaluation of query performance prediction methods by range
SPIRE'10 Proceedings of the 17th international conference on String processing and information retrieval
Evaluation effort, reliability and reusability in XML retrieval
Journal of the American Society for Information Science and Technology
Mimicking Web search engines for expert search
Information Processing and Management: an International Journal
System effectiveness, user models, and user utility: a conceptual framework for investigation
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Evaluating diversified search results using per-intent graded relevance
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Displacement based unsupervised metric for evaluating rank aggregation
PReMI'11 Proceedings of the 4th international conference on Pattern recognition and machine intelligence
Toward a fair review-management system
ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part II
Discounted cumulative gain and user decision models
SPIRE'11 Proceedings of the 18th international conference on String processing and information retrieval
Search Engine Query Clustering Using Top-k Search Results
WI-IAT '11 Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01
Fidelity metrics for estimation models
Proceedings of the International Conference on Computer-Aided Design
Dynamic rank correlation computing for financial risk analysis
KSEM'11 Proceedings of the 5th international conference on Knowledge Science, Engineering and Management
Evaluation with informational and navigational intents
Proceedings of the 21st international conference on World Wide Web
Query performance prediction based on ranking list dispersion
FDIA'09 Proceedings of the Third BCS-IRSG conference on Future Directions in Information Access
A formal and empirical analysis of the fuzzy gamma rank correlation coefficient
Information Sciences: an International Journal
Social book search: comparing topical relevance judgements and book suggestions for evaluation
Proceedings of the 21st ACM international conference on Information and knowledge management
A comprehensive analysis of parameter settings for novelty-biased cumulative gain
Proceedings of the 21st ACM international conference on Information and knowledge management
A Comparison Study for Novelty Control Mechanisms Applied to Web News Stories
WI-IAT '12 Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01
Summaries, ranked retrieval and sessions: a unified framework for information access evaluation
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
On the measurement of test collection reliability
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
A mutual information-based framework for the analysis of information retrieval systems
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
The impact of intent selection on diversified search evaluation
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Comparison of chemical similarity measures using different numbers of query structures
Journal of Information Science
Diversified search evaluation: lessons from the NTCIR-9 INTENT task
Information Retrieval
Exploiting user disagreement for web search evaluation: an experimental approach
Proceedings of the 7th ACM international conference on Web search and data mining
| Hi-index | 0.00 |
In the field of information retrieval, one is often faced with the problem of computing the correlation between two ranked lists. The most commonly used statistic that quantifies this correlation is Kendall's Τ. Often times, in the information retrieval community, discrepancies among those items having high rankings are more important than those among items having low rankings. The Kendall's Τ statistic, however, does not make such distinctions and equally penalizes errors both at high and low rankings. In this paper, we propose a new rank correlation coefficient, AP correlation (Τap), that is based on average precision and has a probabilistic interpretation. We show that the proposed statistic gives more weight to the errors at high rankings and has nice mathematical properties which make it easy to interpret. We further validate the applicability of the statistic using experimental data.