Comparisons of single- and multiple-hidden-layer neural networks

Authors:
Takehiko Nakama
Affiliations:
European Center for Soft Computing, Edificio Científico Tecnológico, Mieres, Spain
Venue:
ISNN'11 Proceedings of the 8th international conference on Advances in neural networks - Volume Part I
Year:
2011

Citing 11
Cited 0

Multilayer feedforward networks are universal approximators

Neural Networks
On the approximate realization of continuous mappings by neural networks

Neural Networks
Learning internal representations by error propagation

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
Approximation and Estimation Bounds for Artificial Neural Networks

Machine Learning - Special issue on computational learning theory
Fundamentals of neural networks: architectures, algorithms, and applications

Fundamentals of neural networks: architectures, algorithms, and applications
Neural Networks: A Comprehensive Foundation

Neural Networks: A Comprehensive Foundation
Fundamentals of Artificial Neural Networks

Fundamentals of Artificial Neural Networks
The general inefficiency of batch training for gradient descent learning

Neural Networks
Theoretical analysis of batch and on-line training for gradient descent learning in neural networks

Neurocomputing
Some new results on neural network approximation

Neural Networks
A theoretical comparison of batch-mode, on-line, cyclic, and almost-cyclic learning

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this study we conduct fair and systematic comparisons of two types of neural networks: single- and multiple-hidden-layer networks. For fair comparisons, we ensure that the two types use the same activation and output functions and have the same numbers of nodes, feedforward connections, and parameters. The networks are trained by the gradient descent algorithm to approximate linear and quadratic functions, and we examine their convergence properties. We show that, in both linear and quadratic cases, the learning rate is more flexible for networks with a single hidden layer than for those with multiple hidden layers. We also show that single-hidden-layer networks converge faster to linear target functions compared to multiple-hidden-layer networks.