Resizable, scalable, concurrent hash tables via relativistic programming

  • Authors:
  • Josh Triplett;Paul E. McKenney;Jonathan Walpole

  • Affiliations:
  • Portland State University;IBM Linux Technology Center;Portland State University

  • Venue:
  • USENIXATC'11 Proceedings of the 2011 USENIX conference on USENIX annual technical conference
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present algorithms for shrinking and expanding a hash table while allowing concurrent, wait-free, linearly scalable lookups. These resize algorithms allow Read-Copy Update (RCU) hash tables to maintain constant-time performance as the number of entries grows, and reclaim memory as the number of entries decreases, without delaying or disrupting readers. We call the resulting data structure a relativistic hash table. Benchmarks of relativistic hash tables in the Linux kernel show that lookup scalability during resize improves 125x over reader-writer locking, and 56% over Linux's current state of the art. Relativistic hash lookups experience no performance degradation during a resize. Applying this algorithm to memcached removes a scalability limit for get requests, allowing memcached to scale linearly and service up to 46% more requests per second. Relativistic hash tables demonstrate the promise of a new concurrent programming methodology known as relativistic programming. Relativistic programming makes novel use of existing RCU synchronization primitives, namely the wait-for-readers operation that waits for unfinished readers to complete. This operation, conventionally used to handle reclamation, here allows ordering of updates without read-side synchronization or memory barriers.