Measuring the Search Effectiveness of a Breadth-First Crawl

  • Authors:

  • Dennis Fetterly;Nick Craswell;Vishwa Vinay

  • Affiliations:

  • Microsoft Research Silicon Valley, Mountain View, USA;Microsoft Research Cambridge, Cambridge, UK;Microsoft Research Cambridge, Cambridge, UK

  • Venue:
  • ECIR '09 Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Previous scalability experiments found that early precision improves as collection size increases. However, that was under the assumption that a collection's documents are all sampled with uniform probability from the same population. We contrast this to a large breadth-first web crawl, an important scenario in real-world Web search, where the early documents have quite different characteristics from the later documents. Having observed that NDCG@100 (measured over a set of reference queries) begins to plateau in the initial stages of the crawl, we investigate a number of possible reasons for this behaviour. These include the web-pages themselves, the metric used to measure retrieval effectiveness as well as the set of relevance judgements used.