Histogram-aware sorting for enhanced word-aligned compression in bitmap indexes

  • Authors:
  • Owen Kaser;Daniel Lemire;Kamel Aouiche

  • Affiliations:
  • University of New Brunswick, Saint John, NB, Canada;Université du Québec à Montréal, Montreal, PQ, Canada;Université du Québec à Montréal, Montreal, PQ, Canada

  • Venue:
  • Proceedings of the ACM 11th international workshop on Data warehousing and OLAP
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Bitmap indexes must be compressed to reduce input/output costs and minimize CPU usage. To accelerate logical operations (AND, OR, XOR) over bitmaps, we use techniques based on run-length encoding (RLE), such as Word-Aligned Hybrid (WAH) compression. These techniques are sensitive to the order of the rows: a simple lexicographical sort can divide the index size by 9 and make indexes several times faster. We investigate reordering heuristics based on computed attribute-value histograms. Simply permuting the columns of the table based on these histograms can increase the sorting efficiency by 40%.