The error surface of the simplest xor network has only global minima

Authors:
Ida G. Sprinkhuizen-Kuyper;Egbert J. W. Boers
Affiliations:
Department of Computer Science, Leiden University, Niels Bohrweg 1, 2333 CA, Leiden, The Netherlands;Department of Computer Science, Leiden University, Niels Bohrweg 1, 2333 CA, Leiden, The Netherlands
Venue:
Neural Computation
Year:
1996

Citing 2
Cited 5

Learning internal representations by error propagation

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
Approximation of boolean functions by sigmoidal networks: Part i: xor and other two-variable functions

Neural Computation

The local minima of the error surface of the 2-2-1 XOR network

Annals of Mathematics and Artificial Intelligence
A dynamical model for the analysis and acceleration of learning in feedforward networks

Neural Networks
Neural Network Learning Using Low-Discrepancy Sequence

AI '99 Proceedings of the 12th Australian Joint Conference on Artificial Intelligence: Advanced Topics in Artificial Intelligence
The Early Restart Algorithm

Neural Computation
Effect of refractoriness on learning performance of a pattern sequence

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

The artificial neural network with one hidden unit and the input units connected to the output unit is considered. It is proven that the error surface of this network for the patterns of the XOR problem has minimum values with zero error and that all other stationary points of the error surface are saddlepoints. Also, the volume of the regions in weight space with saddlepoints is zero, hence training this network on the four patterns of the XOR problem using, e.g., backpropagation with momentum, the correct solution with error zero will be reached in the limit with probability one.