Almost Linear VC-Dimension Bounds for Piecewise Polynomial Networks

Authors:
Peter L. Bartlett;Vitaly L. Maiorov;Ron L. Meir
Affiliations:
Department of System Engineering, Australian National University, Canberra, ACT 0200, Australia;Department of Mathematics, Technion, Haifa 32000, Israel;Department of Electrical Engineering,Technion, Haifa 32000, Israel
Venue:
Neural Computation
Year:
1998

Citing 15
Cited 0

Decision theoretic generalizations of the PAC model for neural net and other learning applications

Information and Computation
Finiteness results for sigmoidal “neural” networks

STOC '93 Proceedings of the twenty-fifth annual ACM symposium on Theory of computing
Original Contribution: Multilayer feedforward networks with a nonpolynomial activation function can approximate any function

Neural Networks
Neural nets with superlinear VC-dimension

Neural Computation
Approximation and Estimation Bounds for Artificial Neural Networks

Machine Learning - Special issue on computational learning theory
Sphere packing numbers for subsets of the Boolean n-cube with bounded Vapnik-Chervonenkis dimension

Journal of Combinatorial Theory Series A
Bounding the Vapnik-Chervonenkis Dimension of Concept Classes Parameterized by Real Numbers

Machine Learning - Special issue on COLT '93
Fat-shattering and the learnability of real-valued functions

Journal of Computer and System Sciences
Polynomial bounds for VC dimension of sigmoidal and general Pfaffian neural networks

Journal of Computer and System Sciences - Special issue: dedicated to the memory of Paris Kanellakis
Neural networks with quadratic VC dimension

Journal of Computer and System Sciences - Special issue: dedicated to the memory of Paris Kanellakis
A Theory of Learning and Generalization

A Theory of Learning and Generalization
Neural networks for optimal approximation of smooth and analytic functions

Neural Computation
Efficient agnostic learning of neural networks with bounded fan-in

IEEE Transactions on Information Theory - Part 2
The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

IEEE Transactions on Information Theory
Nonparametric estimation via empirical risk minimization

IEEE Transactions on Information Theory

Quantified Score

Hi-index	0.00

Visualization

Abstract

We compute upper and lower bounds on the VC dimension and pseudodimension of feedforward neural networks composed of piecewise polynomial activation functions. We show that if the number of layers is fixed, then the VC dimension and pseudo-dimension grow as W log W, where W is the number of parameters in the network. This result stands in opposition to the case where the number of layers is unbounded, in which case the VC dimension and pseudo-dimension grow as W2 . We combine our results with recently established approximation error rates and determine error bounds for the problem of regression estimation by piecewise polynomial networks with unbounded weights.