Elements of information theory
Elements of information theory
Bayesian classification (AutoClass): theory and results
Advances in knowledge discovery and data mining
An introduction to Kolmogorov complexity and its applications (2nd ed.)
An introduction to Kolmogorov complexity and its applications (2nd ed.)
Unsupervised Learning of Finite Mixture Models
IEEE Transactions on Pattern Analysis and Machine Intelligence
Kullback proximal algorithms for maximum-likelihood estimation
IEEE Transactions on Information Theory
Using weak supervision in learning Gaussian mixture models
IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
A decision tree-based missing value imputation technique for data pre-processing
AusDM '11 Proceedings of the Ninth Australasian Data Mining Conference - Volume 121
| Hi-index | 0.00 |
The EM algorithm heavily relies on the interpretation of observations as incomplete data but it does not have any control on the uncertainty of missing data. To effectively reduce the uncertainty of missing data, we present a regularized EM algorithm that penalizes the likelihood with the mutual information between the missing data and the incomplete data (or the conditional entropy of the missing data given the observations). The proposed method maintains the advantage of the conventional EM algorithm, such as reliable global convergence, low cost per iteration, economy of storage, and ease of programming. We also apply the regularized EM algorithm to fit the finite mixture model. Our theoretical analysis and experiments show that the new method can efficiently fit the models and effectively simplify over-complicated models.