Bounds on the number of hidden neurons in three-layer binary

Authors:
Zhaozhi Zhang;Xiaomin Ma;Yixian Yang
Affiliations:
Institute of Systems Science, Academia Sinica, Beijing 100080, People's Republic of China;Engineering and Physics Department, Oral Roberts University, Tulsa, OK;Engineering and Physics Department, Oral Roberts University, Tulsa, OK
Venue:
Neural Networks
Year:
2003

Citing 6
Cited 3

Original Contribution: Bounds on the number of hidden units in binary-valued three-layer neural networks

Neural Networks
A unified method to construct neural network decoders for arbitrary codes and decoding rulers

Discrete Mathematics
New binary covering codes obtained by simulated annealing

IEEE Transactions on Information Theory
Several new lower bounds on the size of codes with covering radius one

IEEE Transactions on Information Theory
Enumeration of linear threshold functions from the lattice of hyperplane intersections

IEEE Transactions on Neural Networks
The geometrical learning of binary neural networks

IEEE Transactions on Neural Networks

An efficient approach for building customer profiles from business data

Expert Systems with Applications: An International Journal
Neural network architecture selection: can function complexity help?

Neural Processing Letters
Extension of the generalization complexity measure to real valued input data sets

ISNN'10 Proceedings of the 7th international conference on Advances in Neural Networks - Volume Part I

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper investigates an important problem concerning the complexity of three-layer binary neural networks (BNNs) with one hidden layer. The neuron in the studied BNNs employs a hard limiter activation function with only integer weights and an integer threshold. The studies are focused on implementations of arbitrary Boolean functions which map from {0, 1}n into {0, 1}. A deterministic algorithm called set covering algorithm (SCA) is proposed for the construction of a three-layer BNN to implement an arbitrary Boolean function. The SCA is based on a unit sphere covering (USC) of the Hamming space (HS) which is chosen in advance. It is proved that for the implementation of an arbitrary Boolean function of n-variables (n ≥ 3) by using SCA, ⌊3L/2⌋ hidden neurons are necessary and sufficient, where L is the number of unit spheres contained in the chosen USC of the n-dimensional HS. It is shown that by using SCA, the number of hidden neurons required is much less than that by using a two-parallel hyperplane method. In order to indicate the potential ability of three-layer BNNs, a lower bound on the required number of hidden neurons which is derived by using the method of estimating the Vapnik-Chervonenkis (VC) dimension is also given.