New algorithms on wavelet trees and applications to information retrieval

Authors:
Travis Gagie;Gonzalo Navarro;Simon J. Puglisi
Affiliations:
Department of Computer Science, Aalto University, Finland;Department of Computer Science, University of Chile, Chile;School of Computer Science and Information Technology, Royal Melbourne Institute of Technology, Australia and Department of Informatics, Kings College London, United Kingdom
Venue:
Theoretical Computer Science
Year:
2012

Citing 61
Cited 9

Functional approach to data structures and its use in multidimensional searching

SIAM Journal on Computing
Suffix arrays: a new method for on-line string searches

SIAM Journal on Computing
Stemming algorithms: a case study for detailed evaluation

Journal of the American Society for Information Science - Special issue: evaluation of information retrieval systems
Filtered document retrieval with frequency-sorted indexes

Journal of the American Society for Information Science
Corpus-based stemming using cooccurrence of word variants

ACM Transactions on Information Systems (TOIS)
Exploring the similarity space

ACM SIGIR Forum
Adaptive set intersections, unions, and differences

SODA '00 Proceedings of the eleventh annual ACM-SIAM symposium on Discrete algorithms
Succinct indexable dictionaries with applications to encoding k-ary trees and multisets

SODA '02 Proceedings of the thirteenth annual ACM-SIAM symposium on Discrete algorithms
Adaptive intersection and t-threshold problems

SODA '02 Proceedings of the thirteenth annual ACM-SIAM symposium on Discrete algorithms
Efficient algorithms for document retrieval problems

SODA '02 Proceedings of the thirteenth annual ACM-SIAM symposium on Discrete algorithms
Modern Information Retrieval

Modern Information Retrieval
Adding Compression to Block Addressing Inverted Indexes

Information Retrieval
High-order entropy-compressed text indexes

SODA '03 Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms
Tables

Proceedings of the 16th Conference on Foundations of Software Technology and Theoretical Computer Science
Searching large text collections

Handbook of massive data sets
Scaling and related techniques for geometry problems

STOC '84 Proceedings of the sixteenth annual ACM symposium on Theory of computing
Indexing text using the Ziv-Lempel trie

Journal of Discrete Algorithms - SPIRE 2002
Indexing compressed text

Journal of the ACM (JACM)
Rank/select operations on large alphabets: a tool for text indexing

SODA '06 Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm
Inverted files for text search engines

ACM Computing Surveys (CSUR)
Pruned query evaluation using pre-computed impacts

SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Range mode and range median queries on lists and trees

Nordic Journal of Computing
Succinct suffix arrays based on run-length encoding

Nordic Journal of Computing
Compressed full-text indexes

ACM Computing Surveys (CSUR)
Succinct data structures for flexible text retrieval systems

Journal of Discrete Algorithms
Compressed representations of sequences and full-text indexes

ACM Transactions on Algorithms (TALG)
Efficient document retrieval in main memory

SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Alternation and redundancy analysis of the intersection problem

ACM Transactions on Algorithms (TALG)
Geometric Burrows-Wheeler Transform: Linking Range Searching and Text Indexing

DCC '08 Proceedings of the Data Compression Conference
Space-efficient static trees and graphs

SFCS '89 Proceedings of the 30th Annual Symposium on Foundations of Computer Science
Space-Efficient Algorithms for Document Retrieval

CPM '07 Proceedings of the 18th annual symposium on Combinatorial Pattern Matching
Range Medians

ESA '08 Proceedings of the 16th annual European symposium on Algorithms
Succincter

FOCS '08 Proceedings of the 2008 49th Annual IEEE Symposium on Foundations of Computer Science
Range mode and range median queries in constant time and sub-quadratic space

Information Processing Letters
An experimental investigation of set intersection algorithms for text searching

Journal of Experimental Algorithmics (JEA)
Towards Optimal Range Medians

ICALP '09 Proceedings of the 36th International Colloquium on Automata, Languages and Programming: Part I
Efficient Data Structures for the Orthogonal Range Successor Problem

COCOON '09 Proceedings of the 15th Annual International Conference on Computing and Combinatorics
Succinct Orthogonal Range Search Structures on a Grid with Applications to Text Indexing

WADS '09 Proceedings of the 11th International Symposium on Algorithms and Data Structures
Self-indexed Text Compression Using Straight-Line Programs

MFCS '09 Proceedings of the 34th International Symposium on Mathematical Foundations of Computer Science 2009
Range Quantile Queries: Another Virtue of Wavelet Trees

SPIRE '09 Proceedings of the 16th International Symposium on String Processing and Information Retrieval
Data Structures for Range Median Queries

ISAAC '09 Proceedings of the 20th International Symposium on Algorithms and Computation
Transposition invariant string matching

Journal of Algorithms
Time bounds for selection

Journal of Computer and System Sciences
Compact set representation for information retrieval

SPIRE'07 Proceedings of the 14th international conference on String processing and information retrieval
Implicit compression boosting with applications to self-indexing

SPIRE'07 Proceedings of the 14th international conference on String processing and information retrieval
Improved bounds for range mode and range median queries

SOFSEM'08 Proceedings of the 34th conference on Current trends in theory and practice of computer science
Fully-functional succinct trees

SODA '10 Proceedings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms
A fun application of compact data structures to indexing geographic data

FUN'10 Proceedings of the 5th international conference on Fun with algorithms
Top-k ranked document search in general text databases

ESA'10 Proceedings of the 18th annual European conference on Algorithms: Part II
String retrieval for multi-pattern queries

SPIRE'10 Proceedings of the 17th international conference on String processing and information retrieval
Colored range queries and document retrieval

SPIRE'10 Proceedings of the 17th international conference on String processing and information retrieval
Dual-sorted inverted lists

SPIRE'10 Proceedings of the 17th international conference on String processing and information retrieval
Towards optimal range medians

Theoretical Computer Science
Faster adaptive set intersections for text searching

WEA'06 Proceedings of the 5th international conference on Experimental Algorithms
Compact rich-functional binary relation representations

LATIN'10 Proceedings of the 9th Latin American conference on Theoretical Informatics
Range selection and median: tight cell probe lower bounds and adaptive data structures

Proceedings of the twenty-second annual ACM-SIAM symposium on Discrete Algorithms
Approximate range mode and range median queries

STACS'05 Proceedings of the 22nd annual conference on Theoretical Aspects of Computer Science
Experimental analysis of a fast intersection algorithm for sorted sequences

SPIRE'05 Proceedings of the 12th international conference on String Processing and Information Retrieval
Position-Restricted substring searching

LATIN'06 Proceedings of the 7th Latin American conference on Theoretical Informatics
Finding patterns in given intervals

MFCS'07 Proceedings of the 32nd international conference on Mathematical Foundations of Computer Science
A new succinct representation of RMQ-information and improvements in the enhanced suffix array

ESCAPE'07 Proceedings of the First international conference on Combinatorics, Algorithms, Probabilistic and Experimental Methodologies

The wavelet trie: maintaining an indexed sequence of strings in compressed space

PODS '12 Proceedings of the 31st symposium on Principles of Database Systems
Efficient in-memory top-k document retrieval

SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Wavelet trees for all

CPM'12 Proceedings of the 23rd Annual conference on Combinatorial Pattern Matching
The wavelet matrix

SPIRE'12 Proceedings of the 19th international conference on String Processing and Information Retrieval
Colored range queries and document retrieval

Theoretical Computer Science
Space-efficient data-analysis queries on grids

Theoretical Computer Science
Spaces, Trees, and Colors: The algorithmic landscape of document retrieval on sequences

ACM Computing Surveys (CSUR)
Compact binary relation representations with rich functionality

Information and Computation
Wavelet trees for all

Journal of Discrete Algorithms

Quantified Score

Hi-index	5.23

Visualization

Abstract

Wavelet trees are widely used in the representation of sequences, permutations, text collections, binary relations, discrete points, and other succinct data structures. We show, however, that this still falls short of exploiting all of the virtues of this versatile data structure. In particular we show how to use wavelet trees to solve fundamental algorithmic problems such as range quantile queries, range next value queries, and range intersection queries. We explore several applications of these queries in Information Retrieval, in particular document retrieval in hierarchical and temporal documents, and in the representation of inverted lists.