Indexing data-oriented overlay networks

  • Authors:
  • Karl Aberer;Anwitaman Datta;Manfred Hauswirth;Roman Schmidt

  • Affiliations:
  • Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland;Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland;Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland;Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland

  • Venue:
  • VLDB '05 Proceedings of the 31st international conference on Very large data bases
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

The application of structured overlay networks to implement index structures for data-oriented applications such as peer-to-peer databases or peer-to-peer information retrieval, requires highly efficient approaches for overlay construction, as changing application requirements frequently lead to re-indexing of the data and hence (re)construction of overlay networks. This problem has so far not been addressed in the literature and thus we describe an approach for the efficient construction of data-oriented, structured overlay networks from scratch in a self-organized way. Standard maintenance algorithms for overlay networks cannot accomplish this efficiently, as they are inherently sequential. Our proposed algorithm is completely decentralized, parallel, and can construct a new overlay network with short latency. At the same time it ensures good load-balancing for skewed data key distributions which result from preserving key order relationships as necessitated by data-oriented applications. We provide both a theoretical analysis of the basic algorithms and a complete system implementation that has been tested on PlanetLab. We use this implementation to support peer-to-peer information retrieval and database applications.