Empirical Evidence for Ultrametric Structure in Multi-layer Perceptron Error Surfaces

Authors:
Marcus Gallagher;Tom Downs;Ian Wood
Affiliations:
School of Computer Science and Electrical Engineering, University of Queensland, Brisbane 4072, Australia. e-mail: marcusg@csee.uq.edu.au;School of Computer Science and Electrical Engineering, University of Queensland, Brisbane 4072, Australia.;School of Computer Science and Electrical Engineering, University of Queensland, Brisbane 4072, Australia.
Venue:
Neural Processing Letters
Year:
2002

Citing 6
Cited 0

Introduction to the theory of neural computation

Introduction to the theory of neural computation
How to solve the N bit encoder problem with just two hidden units

Neural Computation
On the geometry of feedforward neural network error surfaces

Neural Computation
XOR has no local minima: a case study in neural network error surface analysis

Neural Networks
Fractal strategies for neural network scaling

The handbook of brain theory and neural networks
The lack of a priori distinctions between learning algorithms

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

Combinatorial optimization problems share an interesting property with spin glass systems in that their state spaces can exhibit ultrametric structure. We use sampling methods to analyse the error surfaces of feedforward multi-layer perceptron neural networks learning encoder problems. The third order statistics of these points of attraction are examined and found to be arranged in a highly ultrametric way. This is a unique result for a finite, continuous parameter space. The implications of this result are discussed.