The mathematics of divergence based online learning in vector quantization

Authors:
Thomas Villmann;Sven Haase;Frank-Michael Schleif;Barbara Hammer;Michael Biehl
Affiliations:
Department of Mathematics/Natural Sciences/Informatics, University of Applied Sciences Mittweida, Mittweida, Germany;Department of Mathematics/Natural Sciences/Informatics, University of Applied Sciences Mittweida, Mittweida, Germany;Institute of Computer Science, Clausthal University of Technology, Clausthal-Zellerfeld, Germany;Institute of Computer Science, Clausthal University of Technology, Clausthal-Zellerfeld, Germany;Johann Bernoulli Inst. for Mathematics and Computer Science, Rijksuniversity Groningen, The Netherlands
Venue:
ANNPR'10 Proceedings of the 4th IAPR TC3 conference on Artificial Neural Networks in Pattern Recognition
Year:
2010

Citing 15
Cited 1

Numerical comparison of the RFCM and AP algorithms for clustering relational data

Pattern Recognition
Self-organizing maps

Self-organizing maps
Kernel-based topographic map formation achieved with an information-theoretic approach

Neural Networks - New developments in self-organizing maps
Generalized relevance learning vector quantization

Neural Networks - New developments in self-organizing maps
Kernel Methods for Pattern Analysis

Kernel Methods for Pattern Analysis
A Novel Kernel Prototype-Based Learning Algorithm

ICPR '04 Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 4 - Volume 04
Relative information of type s, Csiszár's f-divergence, and information inequalities

Information Sciences—Informatics and Computer Science: An International Journal
Neural maps in remote sensing image analysis

Neural Networks - 2003 Special issue: Neural network analysis of complex scientific data: Astronomy and geosciences
Batch and median neural gas

Neural Networks - 2006 Special issue: Advances in self-organizing maps--WSOM'05
Clustering with Bregman Divergences

The Journal of Machine Learning Research
Robust parameter estimation with a small bias against heavy contamination

Journal of Multivariate Analysis
Bregman Divergences and the Self Organising Map

IDEAL '08 Proceedings of the 9th International Conference on Intelligent Data Engineering and Automated Learning
Adaptive relevance matrices in learning vector quantization

Neural Computation
Representation of functional data in neural networks

Neurocomputing
Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation

Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation

Divergence-based classification in learning vector quantization

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose the utilization of divergences in gradient descent learning of supervised and unsupervised vector quantization as an alternative for the squared Euclidean distance. The approach is based on the determination of the Fréchet-derivatives for the divergences, wich can be immediately plugged into the online-learning rules. We provide the mathematical foundation of the respective framework. This framework includes usual gradient descent learning of prototypes as well as parameter optimization and relevance learning for improvement of the performance.