The significance of the Cranfield tests on index languages
SIGIR '91 Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval
Using statistical testing in the evaluation of retrieval experiments
SIGIR '93 Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval
Efficient construction of large test collections
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
Variations in relevance judgments and the measurement of retrieval effectiveness
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
Does “authority” mean quality? predicting expert quality ratings of Web documents
SIGIR '00 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval
Query-based sampling of text databases
ACM Transactions on Information Systems (TOIS)
Evaluation by highly relevant documents
Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval
Adaptive on-line page importance computation
WWW '03 Proceedings of the 12th international conference on World Wide Web
A unified model for metasearch, pooling, and system evaluation
CIKM '03 Proceedings of the twelfth international conference on Information and knowledge management
Forming test collections with no system pooling
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Link analysis ranking: algorithms, theory, and experiments
ACM Transactions on Internet Technology (TOIT)
Crawling a country: better strategies than breadth-first for web page ordering
WWW '05 Special interest tracks and posters of the 14th international conference on World Wide Web
Evaluating implicit feedback models using searcher simulations
ACM Transactions on Information Systems (TOIS)
Efficient PageRank approximation via graph aggregation
Information Retrieval
Distributed query sampling: a quality-conscious approach
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
A statistical method for system evaluation using incomplete judgments
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
ECIR'07 Proceedings of the 29th European conference on IR research
A new rank correlation coefficient for information retrieval
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
On rank correlation and the distance between rankings
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Weighted Rank Correlation in Information Retrieval Evaluation
AIRS '09 Proceedings of the 5th Asia Information Retrieval Symposium on Information Retrieval Technology
The effect of semantic relatedness measures on multi-label classification evaluation
Proceedings of the ACM International Conference on Image and Video Retrieval
A similarity measure for indefinite rankings
ACM Transactions on Information Systems (TOIS)
Time-weighted web authoritative ranking
Information Retrieval
Measures for benchmarking semantic web service matchmaking correctness
ESWC'10 Proceedings of the 7th international conference on The Semantic Web: research and Applications - Volume Part II
Hierarchical link analysis for ranking web data
ESWC'10 Proceedings of the 7th international conference on The Semantic Web: research and Applications - Volume Part II
Dynamic rank correlation computing for financial risk analysis
KSEM'11 Proceedings of the 5th international conference on Knowledge Science, Engineering and Management
A formal and empirical analysis of the fuzzy gamma rank correlation coefficient
Information Sciences: an International Journal
Topic based photo set retrieval using user annotated tags
Multimedia Tools and Applications
| Hi-index | 0.00 |
Some methods for rank correlation in evaluation are examined and their relative advantages and disadvantages are discussed. In particular, it is suggested that different test statistics should be used for providing additional information about the experiments other that the one provided by statistical significance testing. Kendall's τ is often used for testing-rank correlation, yet it is little appropriate if the objective of the test is different from what τ was designed for. In particular, attention should be paid to the null hypothesis. Other measures for rank correlation are described. If one test statistic suggests to reject a hypothesis, other test statistics should be used to support or to revise the decision. The paper then focuses on rank correlation between webpage lists ordered by PageRank for applying the general reflections on these test statistics. An interpretation of PageRank behaviour is provided on the basis of the discussion of the test statistics for rank correlation.