Distribution-aware compressed full-text indexes

Authors:
Paolo Ferragina;Jouni Sirén;Rossano Venturini
Affiliations:
Dept. of Computer Science, Univ. of Pisa;Dept. of Computer Science, Univ. of Helsinki;ISTI, CNR, Pisa
Venue:
ESA'11 Proceedings of the 19th European conference on Algorithms
Year:
2011

Citing 12
Cited 0

The least weight subsequence problem

SIAM Journal on Computing
Computing a minimum weight k-link path in graphs with the concave Monge property

Journal of Algorithms - Special issue on SODA '95 papers
Compressed suffix arrays and suffix trees with applications to text indexing and string matching (extended abstract)

STOC '00 Proceedings of the thirty-second annual ACM symposium on Theory of computing
An analysis of the Burrows—Wheeler transform

Journal of the ACM (JACM)
High-order entropy-compressed text indexes

SODA '03 Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms
Efficient Minimal Perfect Hashing in Nearly Minimal Space

STACS '01 Proceedings of the 18th Annual Symposium on Theoretical Aspects of Computer Science
New text indexing functionalities of the compressed suffix arrays

Journal of Algorithms
Indexing compressed text

Journal of the ACM (JACM)
Compressed full-text indexes

ACM Computing Surveys (CSUR)
Compressed text indexes: From theory to practice

Journal of Experimental Algorithmics (JEA)
On compressing the textual web

Proceedings of the third ACM international conference on Web search and data mining
Mining Query Logs: Turning Search Usage Data into Knowledge

Foundations and Trends in Information Retrieval

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we address the problem of building a compressed self-index that, given a distribution for the pattern queries and a bound on the space occupancy, minimizes the expected query-time within that index-space bound. We solve this problem by exploiting a reduction to the problem of finding a minimum weight K-link path in a particular Directed Acyclic Graph. Interestingly enough, our solution is independent of the underlying compressed index in use. Our experiments compare this optimal strategy with several other standard approaches, showing its effectiveness in practice.