On convergence of the EM algorithmand the Gibbs sampler

Authors:
Sujit K. Sahu;Gareth O. Roberts
Affiliations:
School of Mathematics, University of Wales, Cardiff, Cardiff, CF2 4YH, UK;Department of Mathematics and Statistics, Lancaster University, Lancaster LA1 4YF, UK
Venue:
Statistics and Computing
Year:
1999

Citing 0
Cited 4

An empirical comparison of EM, SEM and MCMC performance for problematic Gaussian mixture likelihoods

Statistics and Computing
Auxiliary mixture sampling with applications to logistic models

Computational Statistics & Data Analysis
Modeling Social Annotation: A Bayesian Approach

ACM Transactions on Knowledge Discovery from Data (TKDD)
A Monte Carlo implementation of the SAGE algorithm for joint soft-multiuser decoding, channel parameter estimation, and code acquisition

IEEE Transactions on Signal Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this article we investigate the relationship between the EM algorithm and the Gibbs sampler. We show that the approximate rate of convergence of the Gibbs sampler by Gaussian approximation is equal to that of the corresponding EM-type algorithm. This helps in implementing either of the algorithms as improvement strategies for one algorithm can be directly transported to the other. In particular, by running the EM algorithm we know approximately how many iterations are needed for convergence of the Gibbs sampler. We also obtain a result that under certain conditions, the EM algorithm used for finding the maximum likelihood estimates can be slower to converge than the corresponding Gibbs sampler for Bayesian inference. We illustrate our results in a number of realistic examples all based on the generalized linear mixed models.