A measure of top-down correlation
Technometrics
The art of computer programming, volume 1 (3rd ed.): fundamental algorithms
The art of computer programming, volume 1 (3rd ed.): fundamental algorithms
Cumulated gain-based evaluation of IR techniques
ACM Transactions on Information Systems (TOIS)
SIAM Journal on Discrete Mathematics
Methods for ranking information retrieval systems without relevance judgments
Proceedings of the 2003 ACM symposium on Applied computing
A study of smoothing methods for language models applied to information retrieval
ACM Transactions on Information Systems (TOIS)
Topic prediction based on comparative retrieval rankings
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Methods for comparing rankings of search engine results
Computer Networks: The International Journal of Computer and Telecommunications Networking - Web dynamics
On rank correlation in information retrieval evaluation
ACM SIGIR Forum
A new rank correlation coefficient for information retrieval
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Rank-biased precision for measurement of retrieval effectiveness
ACM Transactions on Information Systems (TOIS)
On rank correlation and the distance between rankings
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Weighted Rank Correlation in Information Retrieval Evaluation
AIRS '09 Proceedings of the 5th Asia Information Retrieval Symposium on Information Retrieval Technology
Comparing rankings of search results on the Web
Information Processing and Management: an International Journal - Special issue: Infometrics
Space-Limited ranked query evaluation using adaptive pruning
WISE'05 Proceedings of the 6th international conference on Web Information Systems Engineering
Google, bing and a new perspective on ranking similarity
Proceedings of the 20th ACM international conference on Information and knowledge management
AccessRank: predicting what users will do next
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A social node model for realising information dissemination strategies in delay tolerant networks
Proceedings of the 15th ACM international conference on Modeling, analysis and simulation of wireless and mobile systems
Harnessing collective intelligence in social tagging using Delicious
Journal of the American Society for Information Science and Technology
Image re-ranking and rank aggregation based on similarity of ranked lists
Pattern Recognition
"Who's out there?": identifying and ranking lurkers in social networks
Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining
MedRank: discovering influential medical treatments from literature by information network analysis
ADC '13 Proceedings of the Twenty-Fourth Australasian Database Conference - Volume 137
Ranking consistency for image matching and object retrieval
Pattern Recognition
Semantic stability in social tagging streams
Proceedings of the 23rd international conference on World wide web
A scalable re-ranking method for content-based image retrieval
Information Sciences: an International Journal
Hi-index | 0.00 |
Ranked lists are encountered in research and daily life and it is often of interest to compare these lists even when they are incomplete or have only some members in common. An example is document rankings returned for the same query by different search engines. A measure of the similarity between incomplete rankings should handle nonconjointness, weight high ranks more heavily than low, and be monotonic with increasing depth of evaluation; but no measure satisfying all these criteria currently exists. In this article, we propose a new measure having these qualities, namely rank-biased overlap (RBO). The RBO measure is based on a simple probabilistic user model. It provides monotonicity by calculating, at a given depth of evaluation, a base score that is non-decreasing with additional evaluation, and a maximum score that is nonincreasing. An extrapolated score can be calculated between these bounds if a point estimate is required. RBO has a parameter which determines the strength of the weighting to top ranks. We extend RBO to handle tied ranks and rankings of different lengths. Finally, we give examples of the use of the measure in comparing the results produced by public search engines and in assessing retrieval systems in the laboratory.