Matrix analysis
Hierarchical mixtures of experts and the EM algorithm
Neural Computation
Asymptotic Convergence Rate of the EM Algorithm for Gaussian Mixtures
Neural Computation
Adaptive mixtures of local experts
Neural Computation
On convergence properties of the em algorithm for gaussian mixtures
Neural Computation
A Single Loop EM Algorithm for the Mixture of Experts Architecture
ISNN 2009 Proceedings of the 6th International Symposium on Neural Networks: Advances in Neural Networks - Part II
On the correct convergence of the EM algorithm for Gaussian mixtures
Pattern Recognition
IEEE Transactions on Neural Networks
Hi-index | 0.00 |
Mixture of experts (ME) is a modular neural network architecture for supervised classification. The double-loop expectation-maximization (EM) algorithm has been developed for learning the parameters of the ME architecture, and the iteratively reweighted least squares (IRLS) algorithm and the Newton-Raphson algorithm are two popular schemes for learning the parameters in the inner loop or gating network. In this letter, we investigate asymptotic convergence properties of the EM algorithm for ME using either the IRLS or Newton-Raphson approach. With the help of an overlap measure for the ME model, we obtain an upper bound of the asymptotic convergence rate of the EM algorithm in each case. Moreover, we find that for the Newton approach as a specific Newton-Raphson approach to learning the parameters in the inner loop, the upper bound of asymptotic convergence rate of the EM algorithm locally around the true solution Î聵* is , where 脧µ0 is an arbitrarily small number, o(x) means that it is a higher-order infinitesimal as x â聠聮 0, and e(Î聵*) is a measure of the average overlap of the ME model. That is, as the average overlap of the true ME model with large sample tends to zero, the EM algorithm with the Newton approach to learning the parameters in the inner loop tends to be asymptotically superlinear. Finally, we substantiate our theoretical results by simulation experiments.