Neural networks for clustering: theory, architecture, algorithm and neural dynamics

• Authors: Jianhong Wu;Yongqiang Cao
• Venue: Neural networks for clustering: theory, architecture, algorithm and neural dynamics
• Year: 2003

Spectacular advances in information technology and large-scale computing are producing huge and very high dimensional data sets. These data sets arise naturally in a variety of contexts such as text/web mining, bioin-formatics, imaging for diagnostics and surveillance, astronomy and remote sensing. The dimension of these data is in the hundreds or thousands. The traditional clustering algorithms do not work efficiently for data sets in such high dimensional spaces because of the inherent sparsity of data. This is well known as the curse of dimensionality. This dissertation develops a new neural network architecture PART (Projective Adaptive Resonance Theory) and related algorithms based on the neural dynamics to provide a solution to the difficulties in clustering high-dimensional data. The PART architecture is based on the well known ART developed by Carpenter and Grossberg, and a major modification (selective output signaling mechanism) is provided in order to deal with the inherent sparsity of the data points in high dimensional space from many data-mining applications. We provide a rigorous proof of the regular dynamics of the PART model which is a large scale and singularly perturbed system of differential equations coupled with a reset mechanism. Our simulations and comparisons show that the resulting algorithms based on the PART model are effective and efficient in finding projected clusters in high dimensional data sets. In the second part of this dissertation, we propose a provably correct clustering algorithm IMC (Iterative Mean Clustering) and the related mathematical theory. We provide a rigorous proof of the convergence of this algorithm. In particular, in one-cluster case where the data distribution is unimodal, this algorithm converges to the center of the unique cluster starting from an arbitrary initial value. In multi-clusters case where the data distribution is multimodal, this algorithm converges to the center of a cluster that is close to the initial value. Finally, we develop a neural network implementation of the IMC algorithm, called IMC-ART, and introduce a variation of PART algorithm, called PART-A, which combines PART architecture with IMC algorithm.