Super-Linear indices for approximate dictionary searching

  • Authors:

  • Leonid Boytsov

  • Affiliations:

  • North Bethesda, MD

  • Venue:
  • SISAP'12 Proceedings of the 5th international conference on Similarity Search and Applications
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present experimental analysis of approximate search algorithms that involve indexing of deletion neighborhoods. These methods require huge indices whose sizes grow exponentially with respect to the maximum allowable number of errors k. Despite extraordinary space requirements, the super-linear indices are of great interest, because they provide some of the shortest retrieval times. A straightforward implementation that creates a hash index directly over residual strings (obtained by deletions from dictionary words) is not space efficient. Rather than memorizing complete residual strings, we record only deleted characters and their respective positions. These data are indexed using a perfect hash function computed for a set of residual dictionary strings [2]. We carry out an experimental evaluation of this approach against several well-known benchmarks (including FastSS, which stores residual strings directly [3]). Experiments show that our implementation has a comparable or superior performance to that of the fastest benchmarks. At the same time, our implementation requires 4-8 times less space as compared to FastSS.