Dimension reduction for distance-based indexing

  • Authors:
  • Rui Mao;Willard L. Miranker;Daniel P. Miranker

  • Affiliations:
  • Shenzhen University, Shenzhen, Guangdong, China;Yale University, New Haven, CT;University of Texas at Austin, Austin, TX

  • Venue:
  • Proceedings of the Third International Conference on SImilarity Search and APplications
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Distance-based indexing exploits only the triangle inequality to answer similarity queries in metric spaces. Lacking of coordinate structure, mathematical tools in Rn can only be applied indirectly, making it difficult for theoretical study in metric space indexing. Toward solving this problem, we formalize a "pivot space model" where data is mapped from metric space to Rn, preserving all the pair wise distances under Linfin;. With this model, it can be shown that the indexing problem in metric space can be equivalently studied in Rn. Further, we show the necessity of dimension reduction for Rn and that the only effective form of dimension reduction is to select existing dimensions, i.e. pivot selection. The coordinate structure of Rn makes the application of many mathematical tools possible. In particular, Principle Component Analysis (PCA) is incorporated into a heuristic method for pivot selection and shown to be effective over a large range of workloads. We also show that PCA can be used to reliably measure the intrinsic dimension of a metric-space.