On retrieval tests with an inhomogeneous query collection

Authors:
Friedbert Jochum
Affiliations:
Technische Universität Berlin, Franklinstr. 28/29, D-1000, Berlin 10
Venue:
SIGIR '85 Proceedings of the 8th annual international ACM SIGIR conference on Research and development in information retrieval
Year:
1985

Citing 3
Cited 1

Two axioms for evaluation measures in information retrieval

Proc. of the third joint BCS and ACM symposium on Research and development in information retrieval
Information Retrieval Experiment

Information Retrieval Experiment
Measurement-theoretical investigation of the MZ-metric

SIGIR '80 Proceedings of the 3rd annual ACM conference on Research and development in information retrieval

The LIVE-project: retrieval experiments based on evaluation viewpoints

SIGIR '85 Proceedings of the 8th annual international ACM SIGIR conference on Research and development in information retrieval

Quantified Score

Hi-index	0.00

Visualization

Abstract

Retrieval tests are often performed with an inhomogeneous query collection, that means certain properties of queries vary within the collection. This paper investigates under which conditions this inhomogeneity may lead to a loss of measurement precision and validity and should be controlled by an experimental design. In particular, interrelationships between the design of a retrieval test, inhomogeneity of the query collection and measurement theoretical properties of the evaluation measure will be discussed. Based on these results, some recommendations for experimental design of retrieval tests are presented.