Avoiding local minima in feedforward neural networks by simultaneous learning

Authors:
Akarachai Atakulreka;Daricha Sutivong
Affiliations:
Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok, Thailand;Department of Industrial Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok, Thailand
Venue:
AI'07 Proceedings of the 20th Australian joint conference on Advances in artificial intelligence
Year:
2007

Citing 12
Cited 2

On the Problem of Local Minima in Backpropagation

IEEE Transactions on Pattern Analysis and Machine Intelligence
Comparative evaluation of genetic algorithm and backpropagation for training neural networks

Information Sciences—Informatics and Computer Science: An International Journal
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
The local minima-free condition of feedforward neural networks forouter-supervised learning

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Deterministic global optimization for FNN training

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Predicting sun spots using a layered perceptron neural network

IEEE Transactions on Neural Networks
A new evolutionary system for evolving artificial neural networks

IEEE Transactions on Neural Networks
A method to determine the required number of neural-network training repetitions

IEEE Transactions on Neural Networks
A local minimum for the 2-3-1 XOR network

IEEE Transactions on Neural Networks
Smooth function approximation using neural networks

IEEE Transactions on Neural Networks
On the local minima free condition of backpropagation learning

IEEE Transactions on Neural Networks

Letters: Alleviating the problem of local minima in Backpropagation through competitive learning

Neurocomputing
Document-level sentiment classification: An empirical comparison between SVM and ANN

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

Feedforward neural networks are particularly useful in learning a training dataset without prior knowledge. However, weight adjusting with a gradient descent may result in the local minimum problem. Repeated training with random starting weights is among the popular methods to avoid this problem, but it requires extensive computational time. This paper proposes a simultaneous training method with removal criteria to eliminate less promising neural networks, which can decrease the probability of achieving a local minimum while efficiently utilizing resources. The experimental results demonstrate the effectiveness and efficiency of the proposed training method in comparison with conventional training.