5E: A Framework to Yield High Performance in Real-Time Data Mining over the Internet

• Authors: Allan K. Y. Wong;Richard S. L. Wu
• Venue: HPC '00 Proceedings of the The Fourth International Conference on High-Performance Computing in the Asia-Pacific Region-Volume 2 - Volume 2
• Year: 2000

The aim of the 5E (5 essentials) framework proposed in this paper is to yield high performance in real-time data mining over the Internet. The essentials are: a) realize the Internet power with object-based parallelism; b) select an inter-object interaction pattern suitable for the problem; c) apply the correct programming model to cut communication overhead; d) equip program objects with mobility for better system performance; e) work with the correct hardware architecture for timeliness. It was verified by different experiments over the Internet that the 5E is indeed effective for mining association rules from very large databases. These experiments were performed in a stable environment involving a) the Java-based Aglets platform for mobile agents, b) the IBM synthetic data package to generate the necessary large databases, c) the Apriori algorithm for the mining process, and d) the SPDM method for program and data parallelizations. The test results however do not constitute a validation of the proposed 5E framework, but an indication that the research is in the right direction and more investigations would be worthwhile.