Bidirectional Hierarchical Clustering for Web Mining

• Authors: Zhongmei Yao;Ben Choi
• Venue: WI '03 Proceedings of the 2003 IEEE/WIC International Conference on Web Intelligence
• Year: 2003

In this paper we propose a new bidirectional hierarchical clustering system for addressing challenges of web mining. The key feature of our approach is that it aims to maximize the intra-cluster similarity in the bottom-up cluster-merging phase and it ensures to minimize the inter-cluster similarity in the top-down refinement phase. This two-pass approach achieves better clustering than existing one-pass approaches. We also propose a new cluster-merging criterion for allowing more than two clusters to be merged in each step and a new measure of similarity for taking into consideration not only the inter-connectivity between clusters but alsothe internal connectivity within the clusters. These result in reducing the average complexity for creating the final hierarchical structure of clusters from O(n2) to O(n). The hierarchical structure represents a semantic structure between concepts of clusters and is directly applicable to the future of semantic net.