An objective approach to cluster validation

  • Authors:

  • Mohamed Bouguessa;Shengrui Wang;Haojun Sun

  • Affiliations:

  • Department of Computer Science, Faculty of Sciences, University of Sherbrooke, Sherbrooke, Que., Canada J1K 2R1;Department of Computer Science, Faculty of Sciences, University of Sherbrooke, Sherbrooke, Que., Canada J1K 2R1;College of Mathematics and Computer Sciences, Hebei University, Baoding, 071002, China

  • Venue:
  • Pattern Recognition Letters
  • Year:
  • 2006

Quantified Score

Hi-index 0.10

Visualization

Abstract

Cluster validation is a major issue in cluster analysis. Many existing validity indices do not perform well when clusters overlap or there is significant variation in their covariance structure. The contribution of this paper is twofold. First, we propose a new validity index for fuzzy clustering. Second, we present a new approach for the objective evaluation of validity indices and clustering algorithms. Our validity index makes use of the covariance structure of clusters, while the evaluation approach utilizes a new concept of overlap rate that gives a formal measure of the difficulty of distinguishing between overlapping clusters. We have carried out experimental studies using data sets containing clusters of different shapes and densities and various overlap rates, in order to show how validity indices behave when clusters become less and less separable. Finally, the effectiveness of the new validity index is also demonstrated on a number of real-life data sets.