On the performance of feature weighting K-means for text subspace clustering

  • Authors:
  • Liping Jing;Michael K. Ng;Jun Xu;Joshua Zhexue Huang

  • Affiliations:
  • Department of Mathematics, The University of Hong Kong, Hong Kong, China;Department of Mathematics, Hong Kong Baptist University, Hong Kong, China;E-Business Technology Institute, The University of Hong Kong, Hong Kong, China;E-Business Technology Institute, The University of Hong Kong, Hong Kong, China

  • Venue:
  • WAIM'05 Proceedings of the 6th international conference on Advances in Web-Age Information Management
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Text clustering is an effective way of not only organizing textual information, but discovering interesting patterns. Most existing methods, however, suffer from two main drawbacks; they cannot provide an understandable representation for text clusters, and cannot scale to very large text collections. Highly scalable text clustering algorithms are becoming increasingly relevant. In this paper, we present a performance study of a new subspace clustering algorithm for large sparse text data. This algorithm automatically calculates the feature weights in the k-means clustering process. The feature weights are used to discover clusters from subspaces of the text vector space and identify terms that represent the semantics of the clusters. A series of experiments have been conducted to test the performance of the algorithm, including resource consumption and clustering quality. The experimental results on real-world text data have shown that our algorithm quickly converges to a local optimal solution and is scalable to the number of documents, terms and the number of clusters.