Exploring high performance distributed file storage using LDPC codes

  • Authors:
  • Benjamin Gaidioz;Birger Koblitz;Nuno Santos

  • Affiliations:
  • European Organization for Nuclear Research (CERN), 1211 Geneva, Switzerland;European Organization for Nuclear Research (CERN), 1211 Geneva, Switzerland;Ecole Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland

  • Venue:
  • Parallel Computing
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

We explore the feasibility of implementing a reliable, high performance, distributed storage system on a commodity computing cluster. Files are distributed across storage nodes using erasure coding with small low-density parity-check (LDPC) codes, which provide high-reliability with small storage and performance overhead. We present performance measurements done on a prototype system comprising 50 nodes, which are self organised using a peer-to-peer overlay.