Performance analysis of LVQ algorithms: a statistical physics approach

Authors:
Anarta Ghosh;Michael Biehl;Barbara Hammer
Affiliations:
Rijksuniversiteit Groningen, Institute of Mathematics and Computing Science, AV Groningen, The Netherlands;Rijksuniversiteit Groningen, Mathematics and Computing Science, AV Groningen, The Netherlands;Clausthal University of Technology, Institute of Computer Science, Clausthal-Zellerfeld, Germany
Venue:
Neural Networks - 2006 Special issue: Advances in self-organizing maps--WSOM'05
Year:
2006

Citing 10
Cited 4

Self-organizing maps

Self-organizing maps
On-line learning in neural networks

On-line learning in neural networks
Statistical Mechanics of Learning

Statistical Mechanics of Learning
Generalized relevance learning vector quantization

Neural Networks - New developments in self-organizing maps
Soft learning vector quantization

Neural Computation
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
Supervised Neural Gas with General Similarity Measure

Neural Processing Letters
On the Generalization Ability of GRLVQ Networks

Neural Processing Letters
Learning vector quantization: The dynamics of winner-takes-all algorithms

Neurocomputing
Soft nearest prototype classification

IEEE Transactions on Neural Networks

Learning dynamics and robustness of vector quantization and neural gas

Neurocomputing
Adaptive relevance matrices in learning vector quantization

Neural Computation
Window-based example selection in learning vector quantization

Neural Computation
Shared domains of competence of approximate learning models using measures of separability of classes

Information Sciences: an International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

Learning vector quantization (LVQ) constitutes a powerful and intuitive method for adaptive nearest prototype classification. However, original LVQ has been introduced based on heuristics and numerous modifications exist to achieve better convergence and stability. Recently, a mathematical foundation by means of a cost function has been proposed which, as a limiting case, yields a learning rule similar to classical LVQ2.1. It also motivates a modification which shows better stability. However, the exact dynamics as well as the generalization ability of many LVQ algorithms have not been thoroughly investigated so far. Using concepts from statistical physics and the theory of on-line learning, we present a mathematical framework to analyse the performance of different LVQ algorithms in a typical scenario in terms of their dynamics, sensitivity to initial conditions, and generalization ability. Significant differences in the algorithmic stability and generalization ability can be found already for slightly different variants of LVQ. We study five LVQ algorithms in detail: Kohonen's original LVQ1, unsupervised vector quantization (VQ), a mixture of VQ and LVQ, LVQ2.1, and a variant of LVQ which is based on a cost function. Surprisingly, basic LVQ1 shows very good performance in terms of stability, asymptotic generalization ability, and robustness to initializations and model parameters which, in many cases, is superior to recent alternative proposals.