Engineering a multi-purpose test collection for web retrieval experiments

  • Authors:

  • Peter Bailey;Nick Craswell;David Hawking

  • Affiliations:

  • Department of Computer Science, The Australian National University, Canberra, ACT 0209, Australia;CSIRO Mathematics and Information Sciences, GPO Box 664, Canberra, ACT 2601, Australia;CSIRO Mathematics and Information Sciences, GPO Box 664, Canberra, ACT 2601, Australia

  • Venue:
  • Information Processing and Management: an International Journal
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

Past research into text retrieval methods for the Web has been restricted by the lack of a test collection capable of supporting experiments which are both realistic and reproducible. The 1.69 million document WT10g collection is proposed as a multi-purpose testbed for experiments with these attributes, in distributed IR, hyperlink algorithms and conventional ad hoc retrieval.WT10g was constructed by selecting from a superset of documents in such a way that desirable corpus properties were preserved or optimised. These properties include: a high degree of inter-server connectivity, integrity of server holdings, inclusion of documents related to a very wide spread of likely queries, and a realistic distribution of server holding sizes. We confirm that WT10g contains exploitable link information using a site (homepage) finding experiment. Our results show that, on this task, Okapi BM25 works better on propagated link anchor text than on full text.WT10g was used in TREC-9 and TREC-2000 and both topic relevance and homepage finding queries and judgments are available.