A distance-based packing method for high dimensional data

  • Authors:

  • Tae-Wan Kim;Ki-Joune Li

  • Affiliations:

  • Department of Computer Science, Pusan National University, San-30, Jang-Jun Dong, Kum-Jung Gu, Pusan, Korea;Department of Computer Science, Pusan National University, San-30, Jang-Jun Dong, Kum-Jung Gu, Pusan, Korea

  • Venue:
  • ADC '03 Proceedings of the 14th Australasian database conference - Volume 17
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

Minkowski-sum cost model indicates that balanced data pattitioning is not beneficial for high dimensional data. Thus we study several unbalanced partitioning methods and propose cost models for them based on Minkowski-sum cost model. Our cost models indicate that the distance to one of both ends of data space dominates the expected value under uniform data distribution. We generalize studied methods to adapt to data distribution and propose a new partitioning method, called DD-CSP (Distance-based Distribution-adaptive Cyclic Sliced Partition), for high-dimensional index structures. At each partition, it splits data from lower end or higher end to the center of data space based on distance cost function. Based on this fact, we propose a data structure called DSR(Dimension-independent Single value Representation) which takes constant amount of storage to represent MBHs(Minimum Bounding Hyper-cubes) independent of dimension.In our experimental studies, we compare DD-CSP with R-tree, HP, STR, TGS, and methods analyzed in our paper on real and synthetic data sets with wide ranges of dimensions and of selectivities varying from 10-1 to 10-9. In all experiments, we show that our method, DD-CSP, outperforms all other methods and achieves up to 567% savings in response time. Thus it is a clearly winning strategy in terms of range queries and storage requirements.