A model selection method based on bound of learning coefficient

Authors:
Keisuke Yamazaki;Kenji Nagata;Sumio Watanabe;Klaus-Robert Müller
Affiliations:
Precision and Intelligence Laboratory, Tokyo Institute of Technology, Yokohama, Japan;Dept. of Computational Intelligence and Systems Science, Tokyo Institute of Technology, Yokohama, Japan;Precision and Intelligence Laboratory, Tokyo Institute of Technology, Yokohama, Japan;Fraunhofer FIRST, IDA, Berlin, Germany
Venue:
ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part II
Year:
2006

Citing 3
Cited 1

Algebraic Analysis for Nonidentifiable Learning Machines

Neural Computation
Stochastic complexity of bayesian networks

UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Singularities in complete bipartite graph-type Boltzmann machines and upper bounds of stochastic complexities

IEEE Transactions on Neural Networks

Asymptotic analysis of Bayesian generalization error with Newton diagram

Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

To decide the optimal size of learning machines is a central issue in the statistical learning theory, and that is why some theoretical criteria such as the BIC are developed. However, they cannot be applied to singular machines, and it is known that many practical learning machines e.g. mixture models, hidden Markov models, and Bayesian networks, are singular. Recently, we proposed the Singular Information Criterion (SingIC), which allows us to select the optimal size of singular machines. The SingIC is based on the analysis of the learning coefficient. So, the machines, to which the SingIC can be applied, are still limited. In this paper, we propose an extension of this criterion, which enables us to apply it to many singular machines, and evaluate the efficiency in Gaussian mixtures. The results offer an effective strategy to select the optimal size.