The local minima of the error surface of the 2-2-1 XOR network

Authors:
Ida G. Sprinkhuizen-Kuyper;Egbert J. W. Boers
Affiliations:
Department of Computer Science, Leiden University, Leiden, The Netherlands E-mail: kuyper@wi.leidenuniv.nl;Department of Computer Science, Leiden University, Leiden, The Netherlands E-mail: kuyper@wi.leidenuniv.nl
Venue:
Annals of Mathematics and Artificial Intelligence
Year:
1999

Citing 4
Cited 2

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations
XOR has no local minima: a case study in neural network error surface analysis

Neural Networks
The error surface of the 2-2-1 XOR network: the finite stationary points

Neural Networks
The error surface of the simplest xor network has only global minima

Neural Computation

Quantified Score

Hi-index	0.01

Visualization

Abstract

All local minima of the error surface of the 2‐2‐1 XOR network are described. A local minimum is defined as a point such that all points in a neighbourhood have an error value greater than or equal to the error value in that point. It is proved that the error surface of the two‐layer XOR network with two hidden units has a number of regions with local minima. These regions of local minima occur for combinations of the weights from the inputs to the hidden nodes such that one or both hidden nodes are saturated for at least two patterns. However, boundary points of these regions of local minima are saddle points. It will be concluded that from each finite point in weight space a strictly decreasing path exists to a point with error zero. This also explains why experiments using higher numerical precision find less “local minima”.