Circumventing Server Bottlenecks: Indirect Large-Scale P2P Data Collection

  • Authors:
  • Di Niu;Baochun Li

  • Affiliations:
  • -;-

  • Venue:
  • ICDCS '08 Proceedings of the 2008 The 28th International Conference on Distributed Computing Systems
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

In most large-scale peer-to-peer (P2P) applications, it is necessary to collect vital statistics data — sometimes referred to as logs — from up to millions of peers. Traditional solutions involve sending large volumes of such data to centralized logging servers, which are not scalable. In addition, they may not be able to retrieve statistics data from departed peers in dynamic peer-to-peer systems. In this paper, we solve this dilemma through an indirect collection mechanism that distributes data using random network coding across the network, from which servers proactively pull such statistics. By buffering data in a decentralized fashion with only a small portion of peer resources, we show that our new mechanism provides a "buffering" zone and a "smoothing" factor to collect large volumes of statistics, with appropriate resilience to peer dynamics and scalability to a large peer population.