Singularities in complete bipartite graph-type Boltzmann machines and upper bounds of stochastic complexities

Authors:
K. Yamazaki;S. Watanabe
Affiliations:
Precision & Intelligence Lab., Tokyo Inst. of Technol., Yokohama, Japan;-
Venue:
IEEE Transactions on Neural Networks
Year:
2005

Citing 0
Cited 8

Resolution of Singularities and Stochastic Complexity of Complete Bipartite Graph-Type Spin Model in Bayesian Estimation

MDAI '07 Proceedings of the 4th international conference on Modeling Decisions for Artificial Intelligence
Equations of states in singular statistical estimation

Neural Networks
Asymptotic analysis of Bayesian generalization error with Newton diagram

Neural Networks
Stochastic Complexity and Generalization Error of a Restricted Boltzmann Machine in Bayesian Estimation

The Journal of Machine Learning Research
A model selection method based on bound of learning coefficient

ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part II
Asymptotic behavior of stochastic complexity of complete bipartite graph-type boltzmann machines

ICONIP'06 Proceedings of the 13 international conference on Neural Information Processing - Volume Part I
On equivalent parameter learning in simplified feature space based on Bayesian asymptotic analysis

Neural Networks
A widely applicable Bayesian information criterion

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

It is well known that Boltzmann machines are nonregular statistical models. The set of their parameters for a small size model is an analytic set with singularities in the space of a large size one. The mathematical foundation of their learning is not yet constructed because of these singularities, though they are applied to information engineering. Recently we established a method to calculate the Bayes generalization errors using an algebraic geometric method even if the models are nonregular. This paper clarifies that the upper bounds of generalization errors in Boltzmann machines are smaller than those in regular statistical models.