Indexing very high-dimensional sparse and quasi-sparse vectors for similarity searches

Authors:
Changzhou Wang;X. Sean Wang
Affiliations:
Mathematics and Computing Technology, Phantom Works, The Boeing Company, Bellevue, Washington, USA/ E-mail: changzhou.wang@boeing.com;Department of Information and Software Engineering, George Mason University, Fairfax, Virginia, USA/ E-mail: xywang@gmu.edu
Venue:
The VLDB Journal — The International Journal on Very Large Data Bases
Year:
2001

Citing 25
Cited 4

The R*-tree: an efficient and robust access method for points and rectangles

SIGMOD '90 Proceedings of the 1990 ACM SIGMOD international conference on Management of data
The hB-tree: a multiattribute indexing method with good guaranteed performance

ACM Transactions on Database Systems (TODS)
Fast subsequence matching in time-series databases

SIGMOD '94 Proceedings of the 1994 ACM SIGMOD international conference on Management of data
Fast multiresolution image querying

SIGGRAPH '95 Proceedings of the 22nd annual conference on Computer graphics and interactive techniques
Nearest neighbor queries

SIGMOD '95 Proceedings of the 1995 ACM SIGMOD international conference on Management of data
FastMap: a fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets

SIGMOD '95 Proceedings of the 1995 ACM SIGMOD international conference on Management of data
Adapting a spatial access structure for document representations in vector space

CIKM '96 Proceedings of the fifth international conference on Information and knowledge management
Similarity-based queries for time series data

SIGMOD '97 Proceedings of the 1997 ACM SIGMOD international conference on Management of data
Distance-based indexing for high-dimensional metric spaces

SIGMOD '97 Proceedings of the 1997 ACM SIGMOD international conference on Management of data
The SR-tree: an index structure for high-dimensional nearest neighbor queries

SIGMOD '97 Proceedings of the 1997 ACM SIGMOD international conference on Management of data
Wavelet-based image indexing techniques with partial sketch retrieval capability

IEEE ADL '97 Proceedings of the IEEE international forum on Research and technology advances in digital libraries
A cost model for nearest neighbor search in high-dimensional data space

PODS '97 Proceedings of the sixteenth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems
The pyramid-technique: towards breaking the curse of dimensionality

SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
High-dimensional index structures database support for next decade's applications (tutorial)

SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
Information Retrieval Systems: Theory and Implementation

Information Retrieval Systems: Theory and Implementation
R-trees: a dynamic index structure for spatial searching

SIGMOD '84 Proceedings of the 1984 ACM SIGMOD international conference on Management of data
The TV-tree: an index structure for high-dimensional data

The VLDB Journal — The International Journal on Very Large Data Bases - Spatial Database Systems
Efficient Similarity Search In Sequence Databases

FODO '93 Proceedings of the 4th International Conference on Foundations of Data Organization and Algorithms
Similarity Indexing with the SS-tree

ICDE '96 Proceedings of the Twelfth International Conference on Data Engineering
M-tree: An Efficient Access Method for Similarity Search in Metric Spaces

VLDB '97 Proceedings of the 23rd International Conference on Very Large Data Bases
The R+-Tree: A Dynamic Index for Multi-Dimensional Objects

VLDB '87 Proceedings of the 13th International Conference on Very Large Data Bases
Hilbert R-tree: An Improved R-tree using Fractals

VLDB '94 Proceedings of the 20th International Conference on Very Large Data Bases
Near Neighbor Search in Large Metric Spaces

VLDB '95 Proceedings of the 21th International Conference on Very Large Data Bases
The X-tree: An Index Structure for High-Dimensional Data

VLDB '96 Proceedings of the 22th International Conference on Very Large Data Bases
On Similarity Queries for Time-Series Data: Constraint Specification and Implementation

CP '95 Proceedings of the First International Conference on Principles and Practice of Constraint Programming

High-dimensional nearest neighbor search with remote data centers

Knowledge and Information Systems
A hierarchical bitmap indexing method for content based multimedia retrieval

IMSA'06 Proceedings of the 24th IASTED international conference on Internet and multimedia systems and applications
ISIS: a new approach for efficient similarity search in sparse databases

DASFAA'10 Proceedings of the 15th international conference on Database Systems for Advanced Applications - Volume Part II
Connecting the dots: mass, energy, word meaning, and particle-wave duality

QI'12 Proceedings of the 6th international conference on Quantum Interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

Similarity queries on complex objects are usually translated into searches among their feature vectors. This paper studies indexing techniques for very high-dimensional (e.g., in hundreds) vectors that are sparse or quasi-sparse, i.e., vectors each having only a small number (e.g., ten) of non-zero or significant values. Based on the R-tree, the paper introduces the xS-tree that uses lossy compression of bounding regions to guarantee a reasonable minimum fan-out within the allocated storage space for each node. In addition, the paper studies the performance and scalability of the xS-tree via experiments.