Adaptive distributed indexing for structured peer-to-peer networks

  • Authors:

  • Linh Thai Nguyen;Wai Gen Yee;Ophir Frieder

  • Affiliations:

  • Illinois Institute of Technology, Chicago, IL, USA;Illinois Institute of Technology, Chicago, IL, USA;Georgetown University, Washington D.C., USA

  • Venue:
  • Proceedings of the 17th ACM conference on Information and knowledge management
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Structured peer-to-peer networks support keyword search by building a distributed index over the collective content shared by all peers. Building the index and processing queries involve data transfer among peers, thus it is important to keep both of these activities bandwidth-efficient. However, this goal is difficult to attain, as smaller, less precise indices reduce index building and access costs but increase query processing cost, which potentially increases overall cost. We study the trade-off between indexing cost and query processing cost in a structured peer-to-peer network and propose a cost-reducing, adaptive, distributed indexing technique based on the term distributions in local shared contents and user query logs. Using this information, we reduce costs by tuning the precision of the index. The approach we take is to group local documents and to index the groups instead of either individual documents or entire peer collections. We control total cost by controlling the number and contents of groups. We propose a probabilistic model to estimate the cost of grouping, which allows us to identify the optimal number of groups to be created. In addition, we propose a cost-based distance function to guide the document grouping process. Experimental results show that our adaptive indexing technique reduces cost by up to 47% compared with peer-level grouping and by up to 73% compared with document-level grouping.