Improving deduplication techniques by accelerating remainder calculations

  • Authors:
  • M. Hirsch;S. T. Klein;Y. Toaff

  • Affiliations:
  • IBM-Diligent, Tel Aviv, Israel;Department of Computer Science, Bar Ilan University, Ramat Gan, Israel;IBM-Diligent, Tel Aviv, Israel

  • Venue:
  • Discrete Applied Mathematics
  • Year:
  • 2014

Quantified Score

Hi-index 0.04

Visualization

Abstract

The time efficiency of many storage systems rely critically on the ability to perform a large number of evaluations of certain hashing functions fast enough. The remainder function BmodP, generally applied with a large prime number P, is often used as a building block of such hashing functions, which leads to the need of accelerating remainder evaluations, possibly using parallel processors. We suggest several improvements exploiting the mathematical properties of the remainder function, leading to iterative or hierarchical evaluations. Experimental results show a 2 to 5-fold increase in the processing speed.