Improved compressed indexes for full-text document retrieval

  • Authors:
  • Djamal Belazzougui;Gonzalo Navarro;Daniel Valenzuela

  • Affiliations:
  • LIAFA, Univ. Paris Diderot, Paris 7, France;Department of Computer Science, University of Chile, Chile;Department of Computer Science, University of Chile, Chile

  • Venue:
  • Journal of Discrete Algorithms
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

We give new space/time tradeoffs for compressed indexes that answer document retrieval queries on general sequences. On a collection of D documents of total length n, current approaches require at least |CSA|+O(nlgDlglgD) or 2|CSA|+o(n) bits of space, where CSA is a full-text index. Using monotone minimal perfect hash functions (mmphfs), we give new algorithms for document listing with frequencies and top-k document retrieval using just |CSA|+O(nlglglgD) bits. We also improve current solutions that use 2|CSA|+o(n) bits, and consider other problems such as colored range listing, top-k most important documents, and computing arbitrary frequencies. We give proof-of-concept experimental results that show that using mmphfs may provide relevant practical tradeoffs for document listing with frequencies.