Hyperparameter learning in probabilistic prototype-based models

Authors:
Petra Schneider;Michael Biehl;Barbara Hammer
Affiliations:
University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science, P.O. Box 407, 9700 AK Groningen, The Netherlands;University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science, P.O. Box 407, 9700 AK Groningen, The Netherlands;University of Bielefeld, Faculty of Technology, CITEC, Universiaetsstrasse 23, 33615 Bielefeld, Germany
Venue:
Neurocomputing
Year:
2010

Citing 10
Cited 2

Self-organizing maps

Self-organizing maps
Generalized relevance learning vector quantization

Neural Networks - New developments in self-organizing maps
Soft learning vector quantization

Neural Computation
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
On the Generalization Ability of GRLVQ Networks

Neural Processing Letters
Fuzzy classification by fuzzy labeled neural gas

Neural Networks - 2006 Special issue: Advances in self-organizing maps--WSOM'05
Dynamics and Generalization Ability of LVQ Algorithms

The Journal of Machine Learning Research
Incremental GRLVQ: Learning relevant features for 3D object recognition

Neurocomputing
Distance learning in discriminative vector quantization

Neural Computation
Diversified SVM ensembles for large data sets

ECML'06 Proceedings of the 17th European conference on Machine Learning

Divergence-based classification in learning vector quantization

Neurocomputing
Relational extensions of learning vector quantization

ICONIP'11 Proceedings of the 18th international conference on Neural Information Processing - Volume Part II

Quantified Score

Hi-index	0.01

Visualization

Abstract

We present two approaches to extend Robust Soft Learning Vector Quantization (RSLVQ). This algorithm for nearest prototype classification is derived from an explicit cost function and follows the dynamics of a stochastic gradient ascent. The RSLVQ cost function is defined in terms of a likelihood ratio and involves a hyperparameter which is kept constant during training. We propose to adapt the hyperparameter in the training phase based on the gradient information. Besides, we propose to base the classifier's decision on the value of the likelihood ratio instead of using the distance based classification approach. Experiments on artificial and real life data show that the hyperparameter crucially influences the performance of RSLVQ. However, it is not possible to estimate the best value from the data prior to learning. We show that the proposed variant of RSLVQ is very robust with respect to the initial value of the hyperparameter. The classification approach based on the likelihood ratio turns out to be superior to distance based classification, if local hyperparameters are adapted for each prototype.