Efficient term proximity search with term-pair indexes

  • Authors:

  • Hao Yan;Shuming Shi;Fan Zhang;Torsten Suel;Ji-Rong Wen

  • Affiliations:

  • Polytechnic Institute of New York University, Brooklyn, NY, USA;Microsoft Research Asia, Beijing, China;Nankai University, Tianjin, China;Polytechnic Institute of New York University, Brooklyn, NY, USA;Microsoft Research Asia, Beijing, China

  • Venue:
  • CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

There has been a large amount of research on early termination techniques in web search and information retrieval. Such techniques return the top-k documents without scanning and evaluating the full inverted lists of the query terms. Thus, they can greatly improve query processing efficiency. However, only a limited amount of efficient top-k processing work considers the impact of term proximity, i.e., the distance between term occurrences in a document, which has recently been integrated into a number of retrieval models to improve effectiveness. In this paper, we propose new early termination techniques for efficient query processing for the case where term proximity is integrated into the retrieval model. We propose new index structures based on a term-pair index, and study new document retrieval strategies on the resulting indexes. We perform a detailed experimental evaluation on our new techniques and compare them with the existing approaches. Experimental results on large-scale data sets show that our techniques can significantly improve the efficiency of query processing.