The importance of convexity in learning with squared loss
COLT '96 Proceedings of the ninth annual conference on Computational learning theory
On Fusers that Perform Better than Best Sensor
IEEE Transactions on Pattern Analysis and Machine Intelligence
| Hi-index | 0.00 |
We consider learning on multilayer neural nets with piecewisepolynomial activation functions and a fixed number k ofnumerical inputs. We exhibit arbitrarily large networkarchitectures for which efficient and provably successful learningalgorithms exist in the rather realistic refinement of Valiant'smodel for probably approximately correct learning ("PAC learning")where no a priori assumptions are required about the "targetfunction" (agnostic learning), arbitrary noise is permitted in thetraining sample, and the target outputs as well as the networkoutputs may be arbitrary reals. The number of computation steps ofthe learning algorithm LEARN that we construct is bounded by apolynomial in the bit-length n of the fixed number of inputvariables, in the bound s for the allowed bit-length ofweights, in 1/ε, where ε is some arbitrary givenbound for the true error of the neural net after training, and in1/δ where δ is some arbitrary given bound for theprobability that the learning algorithm fails for a randomly drawntraining sample. However, the computation time of LEARN isexponential in the number of weights of the considered networkarchitecture, and therefore only of interest for neural nets ofsmall size. This article provides details to the previouslypublished extended abstract (Maass 1994).