Local minima in hierarchical structures of complex-valued neural networks

Authors:
Tohru Nitta
Affiliations:
-
Venue:
Neural Networks
Year:
2013

Citing 9
Cited 0

Original Contribution: Uniqueness of the weights for minimal feedforward nets with a given input-output map

Neural Networks
An extension of the back-propagation algorithm to complex numbers

Neural Networks
XOR has no local minima: a case study in neural network error surface analysis

Neural Networks
The error surface of the 2-2-1 XOR network: the finite stationary points

Neural Networks
Local minima and plateaus in hierarchical structures of multilayer perceptions

Neural Networks
Orthogonality of decision boundaries in complex-valued neural networks

Neural Computation
Complex-valued Neural Networks: Utilizing High-dimensional Parameters

Complex-valued Neural Networks: Utilizing High-dimensional Parameters
Exceptional reducibility of complex-valued neural networks

IEEE Transactions on Neural Networks
On the complex backpropagation algorithm

IEEE Transactions on Signal Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Most of local minima caused by the hierarchical structure can be resolved by extending the real-valued neural network to complex numbers. It was proved in 2000 that a critical point of the real-valued neural network with H-1 hidden neurons always gives many critical points of the real-valued neural network with H hidden neurons. These critical points consist of many lines in the parameter space which could be local minima or saddle points. Local minima cause plateaus which have a strong negative influence on learning. However, most of the critical points of complex-valued neural network are saddle points unlike those of the real-valued neural network. This is a prominent property of the complex-valued neural network.