Integrating association rule mining with relational database systems: alternatives and implications

  • Authors:
  • Sunita Sarawagi;Shiby Thomas;Rakesh Agrawal

  • Affiliations:
  • IBM Almaden Research Center, 650 Harry Road, San Jose, CA;Dept. of Computer & Information Science & Engineering, University of Florida, Gainesville and IBM Almaden Research Center, 650 Harry Road, San Jose, CA;IBM Almaden Research Center, 650 Harry Road, San Jose, CA

  • Venue:
  • SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
  • Year:
  • 1998

Quantified Score

Hi-index 0.01

Visualization

Abstract

Data mining on large data warehouses is becoming increasingly important. In support of this trend, we consider a spectrum of architectural alternatives for coupling mining with database systems. These alternatives include: loose-coupling through a SQL cursor interface; encapsulation of a mining algorithm in a stored procedure; caching the data to a file system on-the-fly and mining; tight-coupling using primarily user-defined functions; and SQL implementations for processing in the DBMS. We comprehensively study the option of expressing the mining algorithm in the form of SQL queries using Association rule mining as a case in point. We consider four options in SQL-92 and six options in SQL enhanced with object-relational extensions (SQL-OR). Our evaluation of the different architectural alternatives shows that from a performance perspective, the Cache-Mine option is superior, although the performance of the SQL-OR option is within a factor of two. Both the Cache-Mine and the SQL-OR approaches incur a higher storage penalty than the loose-coupling approach which performance-wise is a factor of 3 to 4 worse than Cache-Mine. The SQL-92 implementations were too slow to qualify as a competitive option. We also compare these alternatives on the basis of qualitative factors like automatic parallelization, development ease, portability and inter-operability.