Optimization of large join queries
SIGMOD '88 Proceedings of the 1988 ACM SIGMOD international conference on Management of data
Optimization of large join queries: combining heuristics and combinatorial techniques
SIGMOD '89 Proceedings of the 1989 ACM SIGMOD international conference on Management of data
Introduction to algorithms
Randomized algorithms for optimizing large join queries
SIGMOD '90 Proceedings of the 1990 ACM SIGMOD international conference on Management of data
Introduction to parallel computing: design and analysis of algorithms
Introduction to parallel computing: design and analysis of algorithms
Optimization of parallel query execution plans in XPRS
Distributed and Parallel Databases - Selected papers from the first international conference on parallel and distributed information systems
Industrial-strength parallel query optimization: issues and lessons
Information Systems
Tabu search optimization of large join queries
EDBT '94 Proceedings of the 4th international conference on extending database technology: Advances in database technology
Scheduling problems in parallel query optimization
PODS '95 Proceedings of the fourteenth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems
Parallelism and its price: a case study of nonstop SQL/MP
ACM SIGMOD Record
Provably efficient scheduling for languages with fine-grained parallelism
Journal of the ACM (JACM)
Scheduling multithreaded computations by work stealing
Journal of the ACM (JACM)
Parallel dynamic programming for solving the string editing problem on a CGM/BSP
Proceedings of the fourteenth annual ACM symposium on Parallel algorithms and architectures
Automatic Partitioning of Parallel Loops with Parallelepiped-Shaped Tiles
IEEE Transactions on Parallel and Distributed Systems
IEEE Transactions on Parallel and Distributed Systems
Measuring the Complexity of Join Enumeration in Query Optimization
VLDB '90 Proceedings of the 16th International Conference on Very Large Data Bases
Estimating compilation time of a query optimizer
Proceedings of the 2003 ACM SIGMOD international conference on Management of data
Software and the Concurrency Revolution
Queue - Multiprocessors
Load Balancing in a Parallel Dynamic Programming Multi-Method Applied to the 0-1 Knapsack Problem
PDP '06 Proceedings of the 14th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing
An Empirical Evaluation ofWork Stealing with Parallelism Feedback
ICDCS '06 Proceedings of the 26th IEEE International Conference on Distributed Computing Systems
VLDB '06 Proceedings of the 32nd international conference on Very large data bases
Locality and parallelism optimization for dynamic programming algorithm in bioinformatics
Proceedings of the 2006 ACM/IEEE conference on Supercomputing
A parallel dynamic programming algorithm on a multi-core architecture
Proceedings of the nineteenth annual ACM symposium on Parallel algorithms and architectures
Dynamic programming strikes back
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Parallelizing query optimization
Proceedings of the VLDB Endowment
Intel threading building blocks
Intel threading building blocks
Autonomic Query Parallelization using Non-dedicated Computers: An Evaluation of Adaptivity Options
ICAC '06 Proceedings of the 2006 IEEE International Conference on Autonomic Computing
Parallel skyline computation on multicore architectures
Information Systems
Database engines on multicores, why parallelize when you can distribute?
Proceedings of the sixth conference on Computer systems
Parallel processing for stepwise generalisation method on multi-core PC cluster
International Journal of Knowledge and Web Intelligence
Hi-index | 0.00 |
The state of the art commercial query optimizers employ cost-based optimization and exploit dynamic programming (DP) to find the optimal query execution plan (QEP) without evaluating redundant sub-plans. The number of alternative QEPs enumerated by the DP query optimizer can increase exponentially, as the number of joins in the query increases. Recently, by exploiting the coming wave of multi-core processor architectures, a state of the art parallel optimization algorithm [14], referred to as PDPsva, has been proposed to parallelize the "time-consuming" DP query optimization process itself. While PDPsva significantly extends the practical use of DP to queries having up to 20-25 tables, it has several limitations: 1) supporting only the size-driven DP enumerator, 2) statically allocating search space, and 3) not fully exploiting parallelism. In this paper, we propose the first generic solution for parallelizing any type of bottom-up optimizer, including the graph-traversal driven type, and for supporting dynamic search allocation and full parallelism. This is a challenging problem, since recently developed, state of art DP optimizers such as DPcpp [21] and DPhyp [22] are very difficult to parallelize due to tangled dependencies in the join pairs they generate. Unless the solution is very carefully devised, a lot of synchronization conflicts are bound to occur. By viewing a serial bottom-up optimizer as one which generates a totally ordered sequence of join pairs in a streaming fashion, we propose a novel concept of dependency-aware reordering, which minimizes waiting time caused by dependencies of join pairs. To maximize parallelism, we also introduce a series of novel performance optimization techniques: 1) pipelining of join pair generation and plan generation; 2) the synchronization-free global MEMO; and 3) threading across dependencies. Through extensive experiments with various query topologies, we show that our solution supports any type of bottom up optimization, achieving linear speedup for each type. Despite the fact that our solution is generic, due to sophisticated optimization techniques, our generic parallel optimizer outperforms PDPsva tailored to size-driven enumeration. Experimental results also show that our solution is much more robust than PDPsva with respect to search space allocation.