What size net gives valid generalization?
Neural Computation
Concrete mathematics: a foundation for computer science
Concrete mathematics: a foundation for computer science
When Are k-Nearest Neighbor and Back Propagation Accurate for Feasible Sized Sets of Examples?
Proceedings of the EURASIP Workshop 1990 on Neural Networks
COLT '95 Proceedings of the eighth annual conference on Computational learning theory
Agnostically Learning Halfspaces
FOCS '05 Proceedings of the 46th Annual IEEE Symposium on Foundations of Computer Science
| Hi-index | 0.00 |
Several recent papers (Gardner and Derrida 1989; Gyrgyi 1990; Sompolinsky et al. 1990) have found, using methods of statistical physics, that a transition to perfect generalization occurs in training a simple perceptron whose weights can only take values 1. We give a rigorous proof of such a phenomena. That is, we show, for = 2.0821, that if at least n examples are drawn from the uniform distribution on {1, 1}n and classified according to a target perceptron wt {1, 1}n as positive or negative according to whether wtx is nonnegative or negative, then the probability is 2(n) that there is any other such perceptron consistent with the examples. Numerical results indicate further that perfect generalization holds for as low as 1.5.