Word clustering with validity indices

  • Authors:
  • Ahmad El Sayed;Julien Velcin;Djamel Zighed

  • Affiliations:
  • ERIC Laboratory, University of Lyon 2;ERIC Laboratory, University of Lyon 2;ERIC Laboratory, University of Lyon 2

  • Venue:
  • Canadian AI'08 Proceedings of the Canadian Society for computational studies of intelligence, 21st conference on Advances in artificial intelligence
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

The goal of any clustering algorithm producing flat partitions of data is to find the optimal clustering solution and the optimal number of clusters. One natural way to reach this goal without the need for parameters, is to involve a validity index in the clustering process, which can lead to an objective selection of the optimal number of clusters. In this paper, we provide two main contributions. Firstly, since validity indices have been mostly studied in small dimensional datasets, we have chosen to evaluate them in a real-world task: agglomerative clustering of words. Secondly, we propose a new context-aware method that aims at enhancing the validity indices usage as stopping criteria in agglomerative algorithms. Experimental results show that the method is a step-forward in using, with more reliability, validity indices as stopping criteria.