A reliable resilient backpropagation method with gradient ascent

Authors:
Xugang Wang;Hongan Wang;Guozhong Dai;Zheng Tang
Affiliations:
Intelligence Engineering Laboratory, Institute of Software, The Chinese Academy of Sciences, Beijing, China;Intelligence Engineering Laboratory, Institute of Software, The Chinese Academy of Sciences, Beijing, China;Intelligence Engineering Laboratory, Institute of Software, The Chinese Academy of Sciences, Beijing, China;Faculty of Engineering, Toyama University, Toyama-shi, Japan
Venue:
ICIC'06 Proceedings of the 2006 international conference on Intelligent computing: Part II
Year:
2006

Citing 9
Cited 0

SuperSAB: fast adaptive back propagation with good scaling properties

Neural Networks
Learning internal representations by error propagation

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
Local minimum escape using thermodynamic properties of neural networks

Neural Networks
On the Problem of Local Minima in Backpropagation

IEEE Transactions on Pattern Analysis and Machine Intelligence
Introduction to artificial neural systems

Introduction to artificial neural systems
Fundamentals of Artificial Neural Networks

Fundamentals of Artificial Neural Networks
Automatic Scaling using Gamma Learning for Feedforward Neural Networks

IWANN '96 Proceedings of the International Workshop on Artificial Neural Networks: From Natural to Artificial Neural Computation
An algorithm of supervised learning for multilayer neural networks

Neural Computation
Training neural networks with additive noise in the desired signal

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

While the Resilient Backpropagation (RPROP) method can be extremely fast in converging to a solution, it suffers from the local minima problem. In this paper, a fast and reliable learning algorithm for multi-layer artificial neural networks is proposed. The learning model has two phases: the RPROP phase and the gradient ascent phase. The repetition of two phases can help the network get out of local minima. The proposed algorithm is tested on some benchmark problems. For all the above problems, the systems are shown to be capable of escaping from the local minima and converge faster than the Backpropagation with momentum algorithm and the simulated annealing techniques.