Communications of the ACM
What size net gives valid generalization?
Neural Computation
A statistical approach to learning and generalization in layered neural networks
COLT '89 Proceedings of the second annual workshop on Computational learning theory
Neural Computation
Stochastic Complexity in Statistical Inquiry Theory
Stochastic Complexity in Statistical Inquiry Theory
DECISION THEORETIC GENERALIZATIONS OF THE PAC MODEL FORNEURAL NET AND OTHER LEARNING APPLICATIONS
DECISION THEORETIC GENERALIZATIONS OF THE PAC MODEL FORNEURAL NET AND OTHER LEARNING APPLICATIONS
Estimation of network parameters in semiparametric stochastic perceptron
Neural Computation
| Hi-index | 0.00 |
For the problem of dividing the space originally partitioned by a blurred boundary, every learning algorithm can make the probability of incorrect prediction of an individual example decrease with the number of training examples t. We address here the question of how the asymptotic form of (t) as well as its limit of convergence reflect the choice of learning algorithms. The error minimum algorithm is found to exhibit rather slow convergence of (t) to its lower bound 0, (t)-0O(t-2/3). Even for the purpose of minimizing prediction error, the maximum likelihood algorithm can be utilized as an alternative. If the true probability distribution happens to be contained in the family of hypothetical functions, then the boundary estimated from the hypothetical distribution function eventually converges to the best choice. Convergence of the prediction error is then (t)-0O(t-1). If the true distribution is not available from the algorithm, however, the boundary generally does not converge to the best choice, but instead (t)-1O(t-1/2), where 1 0 0.