2005 Special Issue: Cross-entropy embedding of high-dimensional data using the neural gas model

  • Authors:
  • Pablo A. Estévez;Cristián J. Figueroa;Kazumi Saito

  • Affiliations:
  • Department of Electrical Engineering, University of Chile, Casilla 412-3, Santiago, Chile;Department of Electrical Engineering, University of Chile, Casilla 412-3, Santiago, Chile;NTT Communication Science Laboratories, 2-4 Hikaridai, Seika, Kyoto 619-0237, Japan

  • Venue:
  • Neural Networks - 2005 Special issue: IJCNN 2005
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

A cross-entropy approach to mapping high-dimensional data into a low-dimensional space embedding is presented. The method allows to project simultaneously the input data and the codebook vectors, obtained with the Neural Gas (NG) quantizer algorithm, into a low-dimensional output space. The aim of this approach is to preserve the relationship defined by the NG neighborhood function for each pair of input and codebook vectors. A cost function based on the cross-entropy between input and output probabilities is minimized by using a Newton-Raphson method. The new approach is compared with Sammon's non-linear mapping (NLM) and the hierarchical approach of combining a vector quantizer such as the self-organizing feature map (SOM) or NG with the NLM recall algorithm. In comparison with these techniques, our method delivers a clear visualization of both data points and codebooks, and it achieves a better mapping quality in terms of the topology preservation measure q"m.