An introduction to variational methods for graphical models
Learning in graphical models
A view of the EM algorithm that justifies incremental, sparse, and other variants
Learning in graphical models
Accelerating EM for Large Databases
Machine Learning
The information geometry of em variants for speech and image processing
The information geometry of em variants for speech and image processing
On the convergence of bound optimization algorithms
UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Predictive discrete latent factor models for large scale dyadic data
Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Cross-validation and aggregated EM training for robust parameter estimation
Computer Speech and Language
A convergence theorem for the fuzzy subspace clustering (FSC) algorithm
Pattern Recognition
Incremental probabilistic latent semantic analysis for automatic question recommendation
Proceedings of the 2008 ACM conference on Recommender systems
Fully Bayesian Joint Model for MR Brain Scan Tissue and Structure Segmentation
MICCAI '08 Proceedings of the 11th International Conference on Medical Image Computing and Computer-Assisted Intervention, Part II
Generative prior knowledge for discriminative classification
Journal of Artificial Intelligence Research
Statistical Parameter Identification of Analog Integrated Circuit Reverse Models
ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part I
MICCAI '09 Proceedings of the 12th International Conference on Medical Image Computing and Computer-Assisted Intervention: Part II
Adaptive iterative detectors for phase-uncertain channels via variational bounding
IEEE Transactions on Communications
Parametric dictionary design for sparse coding
IEEE Transactions on Signal Processing
Semi-supervised Bayesian ARTMAP
Applied Intelligence
A survey of techniques for incremental learning of HMM parameters
Information Sciences: an International Journal
Hi-index | 0.00 |
The EM algorithm is widely used to develop iterative parameter estimation procedures for statistical models. In cases where these procedures strictly follow the EM formulation, the convergence properties of the estimation procedures are well understood. In some instances there are practical reasons to develop procedures that do not strictly fall within the EM framework. We study EM variants in which the E-step is not performed exactly, either to obtain improved rates of convergence, or due to approximations needed to compute statistics under a model family over which E-steps cannot be realized. Since these variants are not EM procedures, the standard (G)EM convergence results do not apply to them. We present an information geometric framework for describing such algorithms and analyzing their convergence properties. We apply this framework to analyze the convergence properties of incremental EM and variational EM. For incremental EM, we discuss conditions under these algorithms converge in likelihood. For variational EM, we show how the E-step approximation prevents convergence to local maxima in likelihood.