A non-linear dimensionality-reduction technique for fast similarity search in large databases

  • Authors:
  • Khanh Vu;Kien A. Hua;Hao Cheng;Sheau-Dong Lang

  • Affiliations:
  • University of Central Florida, Orlando, FL;University of Central Florida, Orlando, FL;University of Central Florida, Orlando, FL;University of Central Florida, Orlando, FL

  • Venue:
  • Proceedings of the 2006 ACM SIGMOD international conference on Management of data
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

To enable efficient similarity search in large databases, many indexing techniques use a linear transformation scheme to reduce dimensions and allow fast approximation. In this reduction approach the approximation is unbounded, so that the approximation volume extends across the dataspace. This causes over-estimation of retrieval sets and impairs performance.This paper presents a non-linear transformation scheme that extracts two important parameters specifying the data. We prove that these parameters correspond to a bounded volume around the search sphere, irrespective of dimensionality. We use a special workspace-mapping mechanism to derive tight bounds for the parameters and to prove further results, as well as highlighting insights into the problems and our proposed solutions. We formulate a measure that lower-bounds the Euclidean distance, and discuss the implementation of the technique upon a popular index structure. Extensive experiments confirm the superiority of this technique over recent state-of-the-art schemes.