A Very Fast Method for Clustering Big Text Datasets

  • Authors:
  • Frank Lin;William W. Cohen

  • Affiliations:
  • Carnegie Mellon Unversity, USA, email: {frank,wcohen}@cs.cmu.edu;Carnegie Mellon Unversity, USA, email: {frank,wcohen}@cs.cmu.edu

  • Venue:
  • Proceedings of the 2010 conference on ECAI 2010: 19th European Conference on Artificial Intelligence
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Large-scale text datasets have long eluded a family of particularly elegant and effective clustering methods that exploits the power of pair-wise similarities between data points due to the prohibitive cost, time-and space-wise, in operating on a similarity matrix, where the state-of-the-art is at best quadratic in time and in space. We present an extremely fast and simple method also using the power of all pair-wise similarity between data points, and show through experiments that it does as well as previous methods in clustering accuracy, and it does so with in linear time and space, without sampling data points or sparsifying the similarity matrix.