An efficient compression code for text databases

  • Authors:
  • Nieves R. Brisaboa;Eva L. Iglesias;Gonzalo Navarro;José R. Paramá

  • Affiliations:
  • Database Lab., Univ. da Coruña, Facultade de Informática, A Coruña, Spain;Computer Science Dept., Univ. de Vigo, Escola Superior de Enxeñería Informática, Ourense, Spain;Dept. of Computer Science, Univ. de Chile, Santiago, Chile;Database Lab., Univ. da Coruña, Facultade de Informática, A Coruña, Spain

  • Venue:
  • ECIR'03 Proceedings of the 25th European conference on IR research
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present a new compression format for natural language texts, allowing both exact and approximate search without decompression. This new code -called End-Tagged Dense Code- has some advantages with respect to other compression techniques with similar features such as the Tagged Huffman Code of [Moura et al., ACM TOIS 2000]. Our compression method obtains (i) better compression ratios, (ii) a simpler vocabulary representation, and (iii) a simpler and faster encoding. At the same time, it retains the most interesting features of the method based on the Tagged Huffman Code, i.e., exact search for words and phrases directly on the compressed text using any known sequential pattern matching algorithm, efficient word-based approximate and extended searches without any decoding, and efficient decompression of arbitrary portions of the text. As a side effect, our analytical results give new upper and lower bounds for the redundancy of d-ary Huffman codes.