Towards information retrieval evaluation with reduced and only positive judgements

Authors:
Diego Mollá;David Martinez;Iman Amini
Affiliations:
Macquarie University, Sydney, Australia;NICTA and University of Melbourne, Melbourne, Australia;NICTA and RMIT, Melbourne, Australia
Venue:
Proceedings of the 18th Australasian Document Computing Symposium
Year:
2013

Citing 3
Cited 0

OHSUMED: an interactive retrieval evaluation and new large test collection for research

SIGIR '94 Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval
Reliable information retrieval evaluation with incomplete and biased judgements

SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Introduction to Information Retrieval

Introduction to Information Retrieval

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper proposes a document distance-based approach to automatically expand the number of available relevance judgements when those are limited and reduced to only positive judgements. This may happen, for example, when the only available judgements are extracted from a list of references in a published clinical systematic review. We show that evaluations based on these expanded relevance judgements are more reliable than those using only the initially available judgements. We also show the impact of such an evaluation approach as the number of initial judgements decreases.