On convergence properties of the em algorithm for gaussian mixtures

Authors:
Lei Xu;Michael I. Jordan
Affiliations:
Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139 USA and Department of Computer Science, The Chinese University of Hong Kong, Hong Kong;Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139 USA
Venue:
Neural Computation
Year:
1996

Citing 4
Cited 78

Hierarchical mixtures of experts and the EM algorithm

Neural Computation
Statistical physics, mixtures of distributions, and the EM algorithm

Neural Computation
Information geometry of the EM and em algorithms for neural networks

Neural Networks
Convergence results for the EM approach to mixtures of experts architectures

Neural Networks

A constructive, incremental-learning network for mixture modeling and classification

Neural Computation
Comparative Analysis on Convergence Rates of The EM Algorithm and Its Two Modifications for Gaussian Mixtures

Neural Processing Letters
Mutual Information Theory for Adaptive Mixture Models

IEEE Transactions on Pattern Analysis and Machine Intelligence
Sensor Selection by GMB-REM in Real Robot Position Estimation

Journal of Intelligent and Robotic Systems
Improved Sensor Selection Technique by Integrating Sensor Fusion in Robot Position Estimation

Journal of Intelligent and Robotic Systems
Mixture models of categorization

Journal of Mathematical Psychology
On Convergence of an Iterative Factor Estimate Algorithm for the NFA Model

ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Applying Mutual Information to Adaptive Mixture Models

IDEAL '00 Proceedings of the Second International Conference on Intelligent Data Engineering and Automated Learning, Data Mining, Financial Engineering, and Intelligent Agents
On the Kernel Widths in Radial-Basis Function Networks

Neural Processing Letters
Symbolic Place Recognition in Voronoi-Based Maps by Using Hidden Markov Models

Journal of Intelligent and Robotic Systems
On the Use of Optimization for Data Mining: Theoretical Interactions and eCRM Opportunities

Management Science
Determine the number of components in a mixture model by the extended KS test

Pattern Recognition Letters
Distributed optimization in sensor networks

Proceedings of the 3rd international symposium on Information processing in sensor networks
Genetic-Based EM Algorithm for Learning Gaussian Mixture Models

IEEE Transactions on Pattern Analysis and Machine Intelligence
Integrating K-Means Clustering with a Relational DBMS Using SQL

IEEE Transactions on Knowledge and Data Engineering
Asymptotic Convergence Rate of the EM Algorithm for Gaussian Mixtures

Neural Computation
The Bayesian Evidence Scheme for Regularizing Probability-Density Estimating Neural Networks

Neural Computation
An Extended EM Algorithm for Joint Feature Extraction and Classification in Brain-Computer Interfaces

Neural Computation
EEG signal classification using wavelet feature extraction and a mixture of expert model

Expert Systems with Applications: An International Journal
A unified perspective and new results on RHT computing, mixture based learning, and multi-learner based problem solving

Pattern Recognition
State-Space Models: From the EM Algorithm to a Gradient Approach

Neural Computation
Time-Series Segmentation Using Predictive Modular Neural Networks

Neural Computation
A Probabilistic Analysis of EM for Mixtures of Separated, Spherical Gaussians

The Journal of Machine Learning Research
Gaussian Mean-Shift Is an EM Algorithm

IEEE Transactions on Pattern Analysis and Machine Intelligence
Bayesian networks for imputation in classification problems

Journal of Intelligent Information Systems
A theoretical framework for multiple neural network systems

Neurocomputing
Missing Data Imputation Techniques

International Journal of Business Intelligence and Data Mining
A self-training semi-supervised SVM algorithm and its application in an EEG-based brain computer interface speller system

Pattern Recognition Letters
Estimating local optimums in EM algorithm over Gaussian mixture model

Proceedings of the 25th international conference on Machine learning
GAKREM: A novel hybrid clustering algorithm

Information Sciences: an International Journal
Feature Selection Based on the Rough Set Theory and Expectation-Maximization Clustering Algorithm

RSCTC '08 Proceedings of the 6th International Conference on Rough Sets and Current Trends in Computing
Singularity and Slow Convergence of the EM algorithm for Gaussian Mixtures

Neural Processing Letters
Linear manifold topographic map formation based on an energy function with on-line adaptation rules

Neurocomputing
Probabilistic inference from arbitrary uncertainty using mixtures of factorized generalized gaussians

Journal of Artificial Intelligence Research
A hybrid EM approach to spatial clustering

Computational Statistics & Data Analysis
Online EM algorithm for mixture with application to internet traffic modeling

Computational Statistics & Data Analysis
Learning mixture models via component-wise parameter smoothing

Computational Statistics & Data Analysis
Improved search for socially annotated data

Proceedings of the VLDB Endowment
Articulated object tracking by rendering consistent appearance parts

ICRA'09 Proceedings of the 2009 IEEE international conference on Robotics and Automation
Asymptotic convergence properties of the EM algorithm with respect to the overlap in the mixture

Neurocomputing
On the correct convergence of the EM algorithm for Gaussian mixtures

Pattern Recognition
Semantics-guided clustering of heterogeneous XML schemas

Journal on data semantics IX
Integrating local distribution information with level set for boundary extraction

Journal of Visual Communication and Image Representation
Machine learning problems from optimization perspective

Journal of Global Optimization
Semi-supervised Bayesian ARTMAP

Applied Intelligence
Knowledge discovery from 3D human motion streams through semantic dimensional reduction

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)
Modeling of video sequences by Gaussian mixture: application in motion estimation by block matching method

EURASIP Journal on Advances in Signal Processing
Robust multiple cardiac arrhythmia detection through bispectrum analysis

Expert Systems with Applications: An International Journal
Multibandwidth kernel-based object tracking

Advances in Artificial Intelligence - Special issue on machine learning paradigms for modeling spatial and temporal information in multimedia data mining
Theory and Use of the EM Algorithm

Foundations and Trends in Signal Processing
Robust Heteroscedastic Probabilistic Neural Network for multiple source partial discharge pattern recognition - Significance of outliers on classification capability

Expert Systems with Applications: An International Journal
Asymptotic convergence properties of the em algorithm for mixture of experts

Neural Computation
Sample-weighted clustering methods

Computers & Mathematics with Applications
A method for mixed states texture segmentation with simultaneous parameter estimation

Pattern Recognition Letters
Accelerating EM: an empirical study

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
A two-round variant of EM for Gaussian mixtures

UAI'00 Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence
Gaussian process networks

UAI'00 Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence
A maximum profit coverage algorithm with application to small molecules cluster identification

WEA'06 Proceedings of the 5th international conference on Experimental Algorithms
An approach for clustering semantically heterogeneous XML schemas

OTM'05 Proceedings of the 2005 Confederated international conference on On the Move to Meaningful Internet Systems - Volume >Part I
On the convergence of bound optimization algorithms

UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Autonomous and deterministic probabilistic neural network using global k-means

ISNN'06 Proceedings of the Third international conference on Advances in Neural Networks - Volume Part I
Learning bayesian networks structure with continuous variables

ADMA'06 Proceedings of the Second international conference on Advanced Data Mining and Applications
Minimal RBF networks by gaussian mixture model

ICIC'05 Proceedings of the 2005 international conference on Advances in Intelligent Computing - Volume Part I
Gaussian mixture modeling of histograms for contrast enhancement

Expert Systems with Applications: An International Journal
Manifold constrained finite gaussian mixtures

IWANN'05 Proceedings of the 8th international conference on Artificial Neural Networks: computational Intelligence and Bioinspired Systems
A variable initialization approach to the EM algorithm for better estimation of the parameters of hidden markov model based acoustic modeling of speech signals

ICDM'06 Proceedings of the 6th Industrial Conference on Data Mining conference on Advances in Data Mining: applications in Medicine, Web Mining, Marketing, Image and Signal Mining
A robust EM clustering algorithm for Gaussian mixture models

Pattern Recognition
Sequential minimal optimization in convex clustering repetitions

Statistical Analysis and Data Mining
Modeling spatial dependencies and semantic concepts in data mining

Proceedings of the 3rd International Conference on Computing for Geospatial Research and Applications
A multi-threshold segmentation approach based on Artificial Bee Colony optimization

Applied Intelligence
Gaussian mixture model based retrieval technique for lossy compressed color images

ICIAR'07 Proceedings of the 4th international conference on Image Analysis and Recognition
The WiMap: A Dynamic Indoor WLAN Localization System

International Journal of Advanced Pervasive and Ubiquitous Computing
Adaptive splitting and selection algorithm for classification of breast cytology images

ICCCI'12 Proceedings of the 4th international conference on Computational Collective Intelligence: technologies and applications - Volume Part I
A novel inductive semi-supervised SVM with graph-based self-training

IScIDE'12 Proceedings of the third Sino-foreign-interchange conference on Intelligent Science and Intelligent Data Engineering
Central clustering of categorical data with automated feature weighting

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
On-line EM Variants for Multivariate Normal Mixture Model in Background Learning and Moving Foreground Detection

Journal of Mathematical Imaging and Vision
A survey on latent tree models and applications

Journal of Artificial Intelligence Research
A graph matching algorithm based on concavely regularized convex relaxation

Neurocomputing

Quantified Score

Hi-index	0.02

Visualization

Abstract

We build up the mathematical connection between the “Expectation-Maximization” (EM) algorithm and gradient-based approaches for maximum likelihood learning of finite gaussian mixtures. We show that the EM step in parameter space is obtained from the gradient via a projection matrix P, and we provide an explicit expression for the matrix. We then analyze the convergence of EM in terms of special properties of P and provide new results analyzing the effect that P has on the likelihood surface. Based on these mathematical results, we present a comparative discussion of the advantages and disadvantages of EM and other algorithms for the learning of gaussian mixture models.