Using intrinsic data skew to improve hash join performance

  • Authors:
  • Bryce Cutt;Ramon Lawrence

  • Affiliations:
  • Department of Computer Science, University of British Columbia, Okanagan, 3333 University Way Kelowna, British Columbia, Canada V1V 1V7;Department of Computer Science, University of British Columbia, Okanagan, 3333 University Way Kelowna, British Columbia, Canada V1V 1V7

  • Venue:
  • Information Systems
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Hash join is used to join large, unordered relations and operates independently of the data distributions of the join relations. Real-world data sets are not uniformly distributed and often contain significant skew. Although partition skew has been studied for hash joins, no prior work has examined how exploiting data skew can improve the performance of hash join. In this paper, we present histojoin, a join algorithm that uses histograms to identify data skew and improve join performance. Experimental results show that for skewed data sets histojoin performs significantly fewer I/O operations and is faster by 10-60% than hybrid hash join.