Move-to-Front, Distance Coding, and Inversion Frequencies revisited

  • Authors:
  • Travis Gagie;Giovanni Manzini

  • Affiliations:
  • Department of Computer Science, University of Eastern Piedmont, Italy;Department of Computer Science, University of Eastern Piedmont, Italy

  • Venue:
  • Theoretical Computer Science
  • Year:
  • 2010

Quantified Score

Hi-index 5.23

Visualization

Abstract

Move-to-Front, Distance Coding and Inversion Frequencies are three simple and effective techniques used to process the output of the Burrows-Wheeler Transform. In this paper we provide the first complete comparative analyses of these techniques, establishing upper and lower bounds on their compression ratios. We describe simple variants of these three techniques that compress any string up to a constant factor of its kth-order empirical entropy for any k=0. At the same time we prove lower bounds for the compression of arbitrary strings which show these variants to be nearly optimal. The bounds we establish are ''entropy-only'' bounds in the sense that they do not involve non-constant overheads. Our analyses provide new insights into the inner workings of these techniques, partially explain their good behavior in practice, and suggest strategies for improving their performance.