Split and Merge EM Algorithm for Improving Gaussian Mixture Density Estimates

Authors:
Naonori Ueda;Ryohei Nakano;Zoubin Ghahramani;Geoffrey E. Hinton
Affiliations:
NTT Communication Science Laboratories, Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237, Japan;NTT Communication Science Laboratories, Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237, Japan;Gatsby Computational Neuroscience Unit, University College London, 17 Queen Square, London WC1N 3AR, UK;Gatsby Computational Neuroscience Unit, University College London, 17 Queen Square, London WC1N 3AR, UK
Venue:
Journal of VLSI Signal Processing Systems
Year:
2000

Citing 5
Cited 6

Fundamentals of speech recognition

Fundamentals of speech recognition
A new competitive learning approach based on an equidistortion principle for designing optimal vector quantizers

Neural Networks
Deterministic annealing EM algorithm

Neural Networks
SMEM algorithm for mixture models

Proceedings of the 1998 conference on Advances in neural information processing systems II
SMEM Algorithm for Mixture Models

Neural Computation

Mixture density estimation with group membership functions

Pattern Recognition Letters
Estimation for the number of components in a mixture model using stepwise split-and-merge EM algorithm

Pattern Recognition Letters
Modeling coarticulation in EMG-based continuous speech recognition

Speech Communication
Redefining class definitions using constraint-based clustering: an application to remote sensing of the earth's surface

Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
Decoding stimulus-related information from single-trial EEG responses based on voltage topographies

Pattern Recognition
A novel split-and-merge algorithm for hierarchical clustering of Gaussian mixture models

Applied Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

The EM algorithm for Gaussian mixture models often gets caught in local maxima of the likelihood which involve having too many Gaussians in one part of the space and too few in another, widely separated part of the space. We present a new EM algorithm which performs split and merge operations on the Gaussians to escape from these configurations. This algorithm uses two novel criteria for efficiently selecting the split and merge candidates. Experimental results on synthetic and real data show the effectiveness of using the split and merge operations to improve the likelihood of both the training data and of held-out test data.