Asymptotic Convergence Properties of Entropy Regularized Likelihood Learning on Finite Mixtures with Automatic Model Selection

Authors:
Zhiwu Lu;Xiaoqing Lu;Zhiyuan Ye
Affiliations:
Institute of Computer Science and Technology, Peking University, Beijing 100871, China;Institute of Computer Science and Technology, Peking University, Beijing 100871, China;Institute of Computer Science and Technology, Peking University, Beijing 100871, China
Venue:
ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Advances in Neural Networks
Year:
2007

Citing 4
Cited 0

BYY harmony learning, structural RPCL, and topological self-organizing on mixture models

Neural Networks - New developments in self-organizing maps
A regularized minimum cross-entropy algorithm on mixtures of experts for time series prediction and curve detection

Pattern Recognition Letters
Entropy Regularized Likelihood Learning on Gaussian Mixture: Two Gradient Implementations for Automatic Model Selection

Neural Processing Letters
Letters: An iterative algorithm for entropy regularized likelihood learning on Gaussian mixture with automatic model selection

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

In finite mixture modelling, it is crucial to select the number of components for a data set. We have proposed an entropy regularized likelihood (ERL) learning principle for the finite mixtures to solve this model selection problem under regularization theory. In this paper, we further give an asymptotic analysis of the ERL learning, and find that the global minimization of the ERL function in a simulated annealing way (i.e., the regularization factor is gradually reduced to zero) leads to automatic model selection on the finite mixtures with a good parameter estimation. As compared with the EM algorithm, the ERL learning can go across the local minima of the negative likelihood and keep robust with respect to initialization. The simulation experiments then prove our theoretic analysis.