Discriminative structure selection method of Gaussian Mixture Models with its application to handwritten digit recognition

Authors:
Xuefeng Chen;Xiabi Liu;Yunde Jia
Affiliations:
Beijing Laboratory of Intelligent Information Technology, School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China;Beijing Laboratory of Intelligent Information Technology, School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China;Beijing Laboratory of Intelligent Information Technology, School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China
Venue:
Neurocomputing
Year:
2011

Citing 16
Cited 0

Automatic Feature Generation for Handwritten Digit Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Bayesian classification (AutoClass): theory and results

Advances in knowledge discovery and data mining
Bayesian Approaches to Gaussian Mixture Modeling

IEEE Transactions on Pattern Analysis and Machine Intelligence
Choice of Basis for Laplace Approximation

Machine Learning
Analysis of Class Separation and Combination of Class-Dependent Features for Handwriting Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Unsupervised Learning of Finite Mixture Models

IEEE Transactions on Pattern Analysis and Machine Intelligence
Inference in model-based cluster analysis

Statistics and Computing
Model selection for probabilistic clustering using cross-validatedlikelihood

Statistics and Computing
Fast SVM Training Algorithm with Decomposition on Very Large Data Sets

IEEE Transactions on Pattern Analysis and Machine Intelligence
Maximization of Mutual Information for Offline Thai Handwriting Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
A trainable feature extractor for handwritten digit recognition

Pattern Recognition
Deformation Models for Image Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Inferring parameters and structure of latent variable models by variational bayes

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Automatic Model Complexity Control Using Marginalized Discriminative Growth Functions

IEEE Transactions on Audio, Speech, and Language Processing
On the posterior-probability estimate of the error rate of nonparametric classification rules

IEEE Transactions on Information Theory
Gradient descent learning algorithm overview: a general dynamical systems perspective

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

Model structure selection is currently an open problem in modeling data via Gaussian Mixture Models (GMM). This paper proposes a discriminative method to select GMM structures for pattern classification. We introduce a GMM structure selection criterion based on a discriminative objective function called Soft target based Max-Min posterior Pseudo-probabilities (Soft-MMP). The structure and the parameters of the optimal GMM are estimated simultaneously by seeking the maximum value of Laplace's approximation of the integrated Soft-MMP function. The line search algorithm is employed to solve this optimization problem. We evaluate the proposed GMM structure selection method through the experiments of handwritten digit recognition on the well-known CENPARMI and MNIST digit databases. Our method behaves better than the manual method and the generative counterparts, including Bayesian Information Criterion (BIC), Minimum Description Length (MDL) and AutoClass. Furthermore, to our best knowledge, the digit classifier trained by using our method achieves the best error rate so far on the CENPARMI database and the error rate comparable to the currently best ones on the MNIST database.