Building scalable solutions to distributed computing problems using probabilistic components

  • Authors:
  • Ken Birman;Indranil Gupta

  • Affiliations:
  • -;-

  • Venue:
  • Building scalable solutions to distributed computing problems using probabilistic components
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

The scalability and reliability of a class of probabilistic protocols for distributed systems can be preserved and enriched through certain protocol composition techniques. This thesis discusses a new methodology for designing fault-tolerant and scalable protocols for distributed group communication system middleware. These new “probabilistic protocols” achieve scale and reliability through the use of decentralization, randomization, message redundancy and recovery techniques, and are an alternative to the traditional deterministic protocol design technique. We address several problems that are central to the design of large-scale distributed systems including group membership, variants of reliable multicast, leader election, data aggregation, and resource location and discovery. In this protocol design methodology, a group communication protocol with desired properties is designed by first choosing the appropriate building blocks from a set of basic probabilistic protocols or strategies. The blocks are then glued together using one of several well-defined composition techniques, to generate a protocol that has its correctness, scalability and reliability properties acquired from individual components. We discuss seven types of building blocks and three composition techniques, and show how the probabilistic protocols mentioned above (viz., for the problems of group membership through resource location and discovery) are a subset of the possible combinations. We also study how different composition techniques affect the inheritance or preservation of protocol properties. The probabilistic protocols discussed in this thesis impose overheads on participating members that are either independent of, or grow slowly with, an increase in group size; the growth is usually a polylogarithmic function. The probabilistic reliability obtained under this overhead is either independent of or improves with increasing group size. The reliability degrades gracefully with rising failure rates of end points and message delivery. Increased overhead can be traded for unproved reliability. Probabilistic protocols can be backed up with inexpensive recovery mechanisms that detect and repair their omissions, thus providing deterministic guarantees to a distributed application. The composition techniques studied in this thesis inherit or preserve to a large degree the scale and reliability of component probabilistic protocols.