Hierarchical Clustering Algorithms for Document Datasets

  • Authors:

  • Ying Zhao;George Karypis;Usama Fayyad

  • Affiliations:

  • Department of Computer Science and Engineering and Digital Technology Center and Army HPC Research Center, University of Minnesota, Minneapolis 55455;Department of Computer Science and Engineering and Digital Technology Center and Army HPC Research Center, University of Minnesota, Minneapolis 55455;Department of Computer Science and Engineering and Digital Technology Center and Army HPC Research Center, University of Minnesota, Minneapolis 55455

  • Venue:
  • Data Mining and Knowledge Discovery
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Fast and high-quality document clustering algorithms play an important role in providing intuitive navigation and browsing mechanisms by organizing large amounts of information into a small number of meaningful clusters. In particular, clustering algorithms that build meaningful hierarchies out of large document collections are ideal tools for their interactive visualization and exploration as they provide data-views that are consistent, predictable, and at different levels of granularity. This paper focuses on document clustering algorithms that build such hierarchical solutions and (i) presents a comprehensive study of partitional and agglomerative algorithms that use different criterion functions and merging schemes, and (ii) presents a new class of clustering algorithms called constrained agglomerative algorithms, which combine features from both partitional and agglomerative approaches that allows them to reduce the early-stage errors made by agglomerative methods and hence improve the quality of clustering solutions. The experimental evaluation shows that, contrary to the common belief, partitional algorithms always lead to better solutions than agglomerative algorithms; making them ideal for clustering large document collections due to not only their relatively low computational requirements, but also higher clustering quality. Furthermore, the constrained agglomerative methods consistently lead to better solutions than agglomerative methods alone and for many cases they outperform partitional methods, as well.