Asymptotic analysis of Bayesian generalization error with Newton diagram

Authors:
Keisuke Yamazaki;Miki Aoyagi;Sumio Watanabe
Affiliations:
Precision and Intelligence Laboratory, Tokyo Institute of Technology, R2-5, 4259 Nagatsuta, Midori-ku, Yokohama, 226-8503, Japan;Advanced Research Institute for the Sciences and Humanities, Nihon University, Nihon University Kaikan Daini Bekkan, 12-5, Goban-cho, Chiyoda-ku, Tokyo 102-8251, Japan;Precision and Intelligence Laboratory, Tokyo Institute of Technology, R2-5, 4259 Nagatsuta, Midori-ku, Yokohama, 226-8503, Japan
Venue:
Neural Networks
Year:
2010

Citing 13
Cited 3

A statistical approach to learning and generalization in layered neural networks

COLT '89 Proceedings of the second annual workshop on Computational learning theory
On the problem in model selection of neural network regression in overrealizable scenario

Neural Computation
Singularities in mixture models and upper bounds of stochastic complexity

Neural Networks
Asymptotic Model Selection for Naive Bayesian Networks

The Journal of Machine Learning Research
Algebraic Analysis for Nonidentifiable Learning Machines

Neural Computation
Stochastic complexities of reduced rank regression in Bayesian estimation

Neural Networks
Variational Bayes Solution of Linear Neural Networks and Its Generalization Performance

Neural Computation
Stochastic Complexities of Gaussian Mixtures in Variational Bayesian Approximation

The Journal of Machine Learning Research
Algebraic geometry and stochastic complexity of hidden Markov models

Neurocomputing
A model selection method based on bound of learning coefficient

ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part II
Stochastic complexity of bayesian networks

UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
A decision-theoretic extension of stochastic complexity and its applications to learning

IEEE Transactions on Information Theory
Singularities in complete bipartite graph-type Boltzmann machines and upper bounds of stochastic complexities

IEEE Transactions on Neural Networks

On generalization error of self-organizing map

ICONIP'10 Proceedings of the 17th international conference on Neural information processing: models and applications - Volume Part II
On equivalent parameter learning in simplified feature space based on Bayesian asymptotic analysis

Neural Networks
Learning coefficient of generalization error in bayesian estimation and vandermonde matrix-type singularity

Neural Computation

Quantified Score

Hi-index	0.01

Visualization

Abstract

Statistical learning machines that have singularities in the parameter space, such as hidden Markov models, Bayesian networks, and neural networks, are widely used in the field of information engineering. Singularities in the parameter space determine the accuracy of estimation in the Bayesian scenario. The Newton diagram in algebraic geometry is recognized as an effective method by which to investigate a singularity. The present paper proposes a new technique to plug the diagram in the Bayesian analysis. The proposed technique allows the generalization error to be clarified and provides a foundation for an efficient model selection. We apply the proposed technique to mixtures of binomial distributions.