Effective sampling for mining association rules

  • Authors:
  • Yanrong Li;Raj P. Gopalan

  • Affiliations:
  • Department of Computing, Curtin University of Technology, Bentley, Western Australia;Department of Computing, Curtin University of Technology, Bentley, Western Australia

  • Venue:
  • AI'04 Proceedings of the 17th Australian joint conference on Advances in Artificial Intelligence
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

As discovering association rules in a very large database is time consuming, researchers have developed many algorithms to improve the efficiency Sampling can significantly reduce the cost of mining, since the mining algorithms need to deal with only a small dataset compared to the original database Especially, if data comes as a stream flowing at a faster rate than can be processed, sampling seems to be the only choice How to sample the data and how big the sample size should be for a given error bound and confidence level are key issues for particular data mining tasks In this paper, we derive the sufficient sample size based on central limit theorem for sampling large datasets with replacement This approach requires smaller sample size than that based on the Chernoff bounds and is effective for association rules mining The effectiveness of the method has been evaluated on both dense and sparse datasets.