Efficient Query Evaluation on Large Textual Collections in a Peer-to-Peer Environment

  • Authors:
  • Jiangong Zhang;Torsten Suel

  • Affiliations:
  • Polytechnic University;Polytechnic University

  • Venue:
  • P2P '05 Proceedings of the Fifth IEEE International Conference on Peer-to-Peer Computing
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

We study the problem of evaluating ranked (top-k) queries on textual collections ranging from multiple giga-bytes to terabytes in size. We focus on the case of a global index organization in a highly distributed environment, and consider a class of ranking functions that includes common variants of the Cosine and Okapi measures. The main bottleneck in such a scenario is the amount of communication required during query evaluation. We propose several efficient query evaluation schemes and evaluate their performance. Our results on real search engine query traces and over 120 million web pages show that after careful optimization such queries can be evaluated at a reasonable cost, while challenges remain for even larger collections and more general classes of ranking functions.