Dynamic itemset counting and implication rules for market basket data

  • Authors:
  • Sergey Brin;Rajeev Motwani;Jeffrey D. Ullman;Shalom Tsur

  • Affiliations:
  • Department of Computer Science, Stanford University and R&D Division, Hitachi America Ltd.;Department of Computer Science, Stanford University;Department of Computer Science, Stanford University;R&D Division, Hitachi America Ltd.

  • Venue:
  • SIGMOD '97 Proceedings of the 1997 ACM SIGMOD international conference on Management of data
  • Year:
  • 1997

Quantified Score

Hi-index 0.03

Visualization

Abstract

We consider the problem of analyzing market-basket data and present several important contributions. First, we present a new algorithm for finding large itemsets which uses fewer passes over the data than classic algorithms, and yet uses fewer candidate itemsets than methods based on sampling. We investigate the idea of item reordering, which can improve the low-level efficiency of the algorithm. Second, we present a new way of generating “implication rules,” which are normalized based on both the antecedent and the consequent and are truly implications (not simply a measure of co-occurrence), and we show how they produce more intuitive results than other methods. Finally, we show how different characteristics of real data, as opposed by synthetic data, can dramatically affect the performance of the system and the form of the results.