Maintaining and checking parity in highly available Scalable Distributed Data Structures

  • Authors:
  • Damian Cieslicki;Stefan Schaeckeler;Thomas Schwarz

  • Affiliations:
  • Department of Computer Engineering, Santa Clara University, 500 El Camino Real, Santa Clara, CA 95053, USA;Department of Computer Engineering, Santa Clara University, 500 El Camino Real, Santa Clara, CA 95053, USA;Department of Computer Engineering, Santa Clara University, 500 El Camino Real, Santa Clara, CA 95053, USA

  • Venue:
  • Journal of Systems and Software
  • Year:
  • 2010

Quantified Score

Hi-index 0.01

Visualization

Abstract

Access to data stored in distributed main memory is much faster than access to local disks. Highly available, Scalable Distributed Data Structures (SDDS) utilize this fast access. They counteract the effects of failed or unavailable nodes by storing data redundantly. Since main memory per node is limited, they generate this redundancy by storing parity data calculated with erasure correcting codes instead of using replication. We present here a way to maintain parity that is about 10 times faster than using the traditional 2PC scheme. We also present a scheme that can diagnose a mismatch between parity and user data with very little network traffic.