Exploiting pipeline interruptions for efficient memory allocation

  • Authors:
  • Josep Aguilar-Saborit;Mohammad Jalali;Dave Sharpe;Victor Muntés Mulero

  • Affiliations:
  • IBM, Toronto, ON, Canada;IBM, Toronto, ON, Canada;IBM, Toronto, ON, Canada;Universitat Politècnica de Catalunya, Barcelona, Spain

  • Venue:
  • Proceedings of the 17th ACM conference on Information and knowledge management
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Efficiency of memory-intensive operations is a key factor in obtaining good performance during multi-join query processing. The pipelined execution of these queries forces the operations in the query plan to be processed concurrently. Making a wrong decision regarding the amount of memory allocated for such operations can have a drastic impact on the response time. However, some of the execution algorithms used at run time interrupt the pipelined execution, ensuring that some operations are never executed concurrently. Because of this, it is essential to explore new approaches in order to improve memory exploitation. In this paper, we address the problem of memory allocation for multiple concurrent operations in a query execution plan. First, we study the dynamic needs for memory during the execution time, and define when two operations coexist. Then, we propose a post-optimization phase, which (i) identifies the operations that are concurrently executed at run time and (ii) costs different memory allocation combinations to find a near optimal solution for any type of query execution plan. We have implemented our techniques in the IBM®DB2® Universal Database" (DB2 UDB) showing that they achieve significant execution time improvements compared with previous approaches, especially for multi-join queries accessing large data volumes.