Spinning relations: high-speed networks for distributed join processing

  • Authors:
  • Philip W. Frey;Romulo Goncalves;Martin Kersten;Jens Teubner

  • Affiliations:
  • ETH Zurich;CWI Amsterdam;CWI Amsterdam;ETH Zurich

  • Venue:
  • Proceedings of the Fifth International Workshop on Data Management on New Hardware
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

By leveraging modern networking hardware (RDMA-enabled network cards), we can shift priorities in distributed database processing significantly. Complex and sophisticated mechanisms to avoid network traffic can be replaced by a scheme that takes advantage of the bandwidth and low latency offered by such interconnects. We illustrate this phenomenon with cyclo-join, an efficient join algorithm based on continuously pumping data through a ring-structured network. Our approach is capable of exploiting the resources of all CPUs and distributed main-memory available in the network for processing queries of arbitrary shape and datasets of arbitrary size.