Indexing high-dimensional data for main-memory similarity search

  • Authors:
  • Xiaohui Yu;Junfeng Dong

  • Affiliations:
  • School of Computer Science and Technology, Shandong University, Jinan, Shandong 250101, China and School of Information Technology, York University, Toronto, ON, Canada M3J 1P3;Microsoft Corporation, One Microsoft Way, Redmond, WA 98052-6399, United States

  • Venue:
  • Information Systems
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

As RAM gets cheaper and larger, in-memory processing of data becomes increasingly affordable. In this paper, we propose a novel index structure, the CSR^+-tree, to support efficient high-dimensional similarity search in main memory. We introduce quantized bounding spheres (QBSs) that approximate bounding spheres (BSs) or data points. We analyze the respective pros and cons of both QBSs and the previously proposed quantized bounding rectangles (QBRs), and take the best of both worlds by carefully incorporating both of them into the CSR^+-tree. We further propose a novel distance computation scheme that eliminates the need for decompressing QBSs or QBRs, which results in significant cost savings. We present an extensive experimental evaluation and analysis of the CSR^+-tree, and compare its performance against that of other representative indexes in the literature. Our results show that the CSR^+-tree consistently outperforms other index structures.