Algebraic geometry of singular learning machines and symmetry of generalization and training errors

Authors:
Sumio Watanabe
Affiliations:
P&I Laboratory, Tokyo Institute of Technology, Mailbox R2-5, 4259 Nagatsuta, Midori-ku, Yokohama 226-8503 Japan
Venue:
Neurocomputing
Year:
2005

Citing 5
Cited 1

On the problem in model selection of neural network regression in overrealizable scenario

Neural Computation
Learning coefficients of layered models when the true distribution mismatches the singularities

Neural Computation
Singularities in mixture models and upper bounds of stochastic complexity

Neural Networks
Asymptotic Model Selection for Naive Bayesian Networks

The Journal of Machine Learning Research
Algebraic Analysis for Nonidentifiable Learning Machines

Neural Computation

Determining structural identifiability of parameter learning machines

Neurocomputing

Quantified Score

Hi-index	0.01

Visualization

Abstract

A lot of hierarchical learning machines such as neural networks and normal mixtures are singular learning machines. In such a learning machine, the likelihood function cannot be approximated by any quadratic form, resulting that the conventional statistical theory does not hold. This paper proves the symmetrical property of the generalization and training errors based on the algebraic geometrical method. Firstly, a new parameterization is introduced by applying the resolution of singularities. Secondly, the asymptotic behavior of the likelihood function is clarified based on the empirical process theory. Lastly, the asymptotic forms of the generalization and training errors are derived. The result will be a mathematical foundation of model selection and hypothesis testing in singular learning machines.