Neural Computation
Neural Computation
A universal theorem on learning curves
Neural Networks
An introduction to wavelets
Statistical theory of learning curves under entropic loss criterion
Neural Computation
Approximation and Estimation Bounds for Artificial Neural Networks
Machine Learning - Special issue on computational learning theory
On the Problem in Model Selection of Neural Network Regression in Overrealizable Scenario
IJCNN '00 Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks (IJCNN'00)-Volume 6 - Volume 6
Neural networks for optimal approximation of smooth and analytic functions
Neural Computation
A decision-theoretic extension of stochastic complexity and its applications to learning
IEEE Transactions on Information Theory
Probabilistic design of layered neural networks based on their unified framework
IEEE Transactions on Neural Networks
Algebraic geometrical methods for hierarchical learning machines
Neural Networks
Generalization Error and Training Error at Singularities of Multilayer Perceptrons
IWANN '01 Proceedings of the 6th International Work-Conference on Artificial and Natural Neural Networks: Connectionist Models of Neurons, Learning Processes and Artificial Intelligence-Part I
Network Optimization through Learning and Pruning in Neuromanifold
PRICAI '02 Proceedings of the 7th Pacific Rim International Conference on Artificial Intelligence: Trends in Artificial Intelligence
The subspace information criterion for infinite dimensional hypothesis spaces
The Journal of Machine Learning Research
Asymptotic properties of the Fisher kernel
Neural Computation
Asymptotic Model Selection for Naive Bayesian Networks
The Journal of Machine Learning Research
Difficulty of Singularity in Population Coding
Neural Computation
Singularities Affect Dynamics of Learning in Neuromanifolds
Neural Computation
Part 2: multilayer perceptron and natural gradient learning
New Generation Computing
Stochastic Complexities of Gaussian Mixtures in Variational Bayesian Approximation
The Journal of Machine Learning Research
Asymptotic Bayesian generalization error when training and test distributions are different
Proceedings of the 24th international conference on Machine learning
Stochastic complexity for mixture of exponential families in generalized variational Bayes
Theoretical Computer Science
Dynamics of learning near singularities in layered networks
Neural Computation
Covariate Shift Adaptation by Importance Weighted Cross Validation
The Journal of Machine Learning Research
MDAI '07 Proceedings of the 4th international conference on Modeling Decisions for Artificial Intelligence
Experimental Bayesian Generalization Error of Non-regular Models under Covariate Shift
Neural Information Processing
Experimental Analysis of Exchange Ratio in Exchange Monte Carlo Method
Neural Information Processing
Singularity and Slow Convergence of the EM algorithm for Gaussian Mixtures
Neural Processing Letters
Upper bound for variational free energy of Bayesian networks
Machine Learning
Analysis of Variational Bayesian Matrix Factorization
PAKDD '09 Proceedings of the 13th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining
Radial Basis Function network learning using localized generalization error bound
Information Sciences: an International Journal
Equations of states in singular statistical estimation
Neural Networks
Marginal Likelihood Integrals for Mixtures of Independence Models
The Journal of Machine Learning Research
Generalization error of automatic relevance determination
ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
On a singular point to contribute to a learning coefficient and weighted resolution of singularities
ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Estimation of poles of zeta function in learning theory using Padé approximation
ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Algebraic geometric study of exchange Monte Carlo method
ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Experimental study of ergodic learning curve in hidden Markov models
ICONIP'08 Proceedings of the 15th international conference on Advances in neuro-information processing - Volume Part I
Design of exchange Monte Carlo method for Bayesian learning in normal mixture models
ICONIP'08 Proceedings of the 15th international conference on Advances in neuro-information processing - Volume Part I
The Journal of Machine Learning Research
On generalization error of self-organizing map
ICONIP'10 Proceedings of the 17th international conference on Neural information processing: models and applications - Volume Part II
The Journal of Machine Learning Research
Asymptotic model selection for naive Bayesian networks
UAI'02 Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence
Theoretical Analysis of Bayesian Matrix Factorization
The Journal of Machine Learning Research
A model selection method based on bound of learning coefficient
ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part II
Automated analytic asymptotic evaluation of the marginal likelihood for latent models
UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Stochastic complexity of bayesian networks
UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Stochastic complexity for mixture of exponential families in variational bayes
ALT'05 Proceedings of the 16th international conference on Algorithmic Learning Theory
Analytic equivalence of bayes a posteriori distributions
ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part I
Free energy of stochastic context free grammar on variational bayes
ICONIP'06 Proceedings of the 13 international conference on Neural Information Processing - Volume Part I
Asymptotic behavior of stochastic complexity of complete bipartite graph-type boltzmann machines
ICONIP'06 Proceedings of the 13 international conference on Neural Information Processing - Volume Part I
Localized bayes estimation for non-identifiable models
ICONIP'06 Proceedings of the 13 international conference on Neural Information Processing - Volume Part I
Generalization performance of exchange monte carlo method for normal mixture models
IDEAL'06 Proceedings of the 7th international conference on Intelligent Data Engineering and Automated Learning
Upper bounds for variational stochastic complexities of bayesian networks
IDEAL'06 Proceedings of the 7th international conference on Intelligent Data Engineering and Automated Learning
A widely applicable Bayesian information criterion
The Journal of Machine Learning Research
| Hi-index | 0.00 |
This article clarifies the relation between the learning curve and the algebraic geometrical structure of a nonidentifiable learning machine such as a multilayer neural network whose true parameter set is an analytic set with singular points. By using a concept in algebraic analysis, we rigorously prove that the Bayesian stochastic complexity or the free energy is asymptotically equal to λ1 log n - (m1 - 1) log log n + constant, where n is the number of training samples and λ1 and m1 are the rational number and the natural number, which are determined as the birational invariant values of the singularities in the parameter space. Also we show an algorithm to calculate λ1 and m1 based on the resolution of singularities in algebraic geometry. In regular statistical models, 2λ1 is equal to the number of parameters and m1 = 1 , whereas in nonregular models, such as multilayer networks, 2λ1 is not larger than the number of parameters and m1 ≥ 1. Since the increase of the stochastic complexity is equal to the learning curve or the generalization error, the nonidentifiable learning machines are better models than the regular ones if Bayesian ensemble learning is applied.