Frequent itemset mining with parallel RDBMS

  • Authors:
  • Xuequn Shang;Kai-Uwe Sattler

  • Affiliations:
  • Department of Computer Science, University of Magdeburg, Magdeburg, Germany;Department of Computer Science and Automation, Technical University of Ilmenau

  • Venue:
  • PAKDD'05 Proceedings of the 9th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining
  • Year:
  • 2005

Quantified Score

Hi-index 0.02

Visualization

Abstract

Data mining on large relational databases has gained popularity and its significance is well recognized. However, the performance of SQL based data mining is known to fall behind specialized implementation. We investigate approaches based on SQL for the problem of finding frequent patterns from a transaction table, including an algorithm that we recently proposed, called Ppropad (Parallel PROjection PAttern Discovery). Ppropad successively projects the transaction table into frequent itemsets to avoid making multiple passes over the large original transaction table and generating a huge sets of candidates. We have built a parallel database system with DB2 and made performance evaluation on it. We prove that data mining with SQL can achieve sufficient performance by the utilization of database tuning.