Distributed top-k query processing by exploiting skyline summaries
Distributed and Parallel Databases
Efficient and domain-invariant competitor mining
Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
| Hi-index | 0.00 |
Given a record set D and a query score function F, a top-k query returns k records from D, whose values of function F on their attributes are the highest. In this paper, we investigate the intrinsic connection between top-k queries and dominant relationships between records, and based on which, we propose an efficient layer-based indexing structure, Pareto-Based Dominant Graph (DG), to improve the query efficiency. Specifically, DG is built offline to express the dominant relationship between records and top-k query is implemented as a graph traversal problem, i.e., Traveler algorithm. We prove theoretically that the size of search space (that is the number of retrieved records from the record set to answer top-k query) in our algorithm is directly related to the cardinality of skyline points in the record set (see Theorem 3). Considering I/O cost, we propose cluster-based storage schema to reduce I/O cost in Traveler algorithm. We also propose the cost estimation methods in this paper. Based on cost analysis, we propose an optimization technique, pseudorecord, to further improve the search efficiency. In order to handle the top-k query in the high-dimension record set, we also propose N-Way Traveler algorithm. In order to handle DG maintenance efficiently, we propose “Insertion” and “Deletion” algorithms for DG. Finally, extensive experiments demonstrate that our proposed methods have significant improvement over its counterparts, including both classical and state art of top-k algorithms.