Fast Lossless Compression of Scientific Floating-Point Data

  • Authors:
  • Paruj Ratanaworabhan;Jian Ke;Martin Burtscher

  • Affiliations:
  • Cornell University;Cornell University;Cornell University

  • Venue:
  • DCC '06 Proceedings of the Data Compression Conference
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

In scientific computing environments, large amounts of floating-point data often need to be transferred between computers as well as to and from storage devices. Compression can reduce the number of bits that need to be transferred and stored. However, the runtime overhead due to compression may be undesirable in high-performance settings where short communication latencies and high bandwidths are essential. This paper describes and evaluates a new compression algorithm that is tailored to such environments. It typically compresses numeric floating-point values better and faster than other algorithms do. On our data sets, it achieves compression ratios between 1.2 and 4.2 as well as compression and decompression throughputs between 2.8 and 5.9 million 64-bit double- precision numbers per second on a 3GHz Pentium 4 machine.