Magnified gradient function with deterministic weight modification in adaptive learning

Authors:
Sin-Chun Ng;Chi-Chung Cheung;Shu-Hung Leung
Affiliations:
Sch. of Sci. & Technol., Open Univ. of Hong Kong, China;-;-
Venue:
IEEE Transactions on Neural Networks
Year:
2004

Citing 0
Cited 16

Wavelet feature selection based neural networks with application to the text independent speaker identification

Pattern Recognition
Rapid and brief communication: Wavelet feature domain adaptive noise reduction using learning algorithm for text-independent speaker recognition

Pattern Recognition
Efficient text independent speaker recognition with wavelet feature selection based multilayered neural network using supervised learning algorithm

Pattern Recognition
A new sub-pixel mapping algorithm based on a BP neural network with an observation model

Neurocomputing
Improved Learning Algorithms of SLFN for Approximating Periodic Function

ICIC '08 Proceedings of the 4th international conference on Intelligent Computing: Advanced Intelligent Computing Theories and Applications - with Aspects of Artificial Intelligence
Boundedness and convergence of online gradient method with penalty for feedforward neural networks

IEEE Transactions on Neural Networks
Local coupled feedforward neural network

Neural Networks
The multi-phase method in fast learning algorithms

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
A constrained approximation algorithm by encoding second-order derivative information into feedforward neural networks

ICIC'09 Proceedings of the Intelligent computing 5th international conference on Emerging intelligent computing technology and applications
Injecting Chaos in Feedforward Neural Networks

Neural Processing Letters
A new learning algorithm for function approximation incorporating a priori information into extreme learning machine

ISNN'06 Proceedings of the Third international conference on Advances in Neural Networks - Volume Part I
Addressing the local minima problem by output monitoring and modification algorithms

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part I
Magnified gradient function to improve first-order gradient-based learning algorithms

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part I
Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part I
Back-propagation with diversive curiosity: An automatic conversion from search stagnation to exploration

Applied Soft Computing
Harnessing chaotic activation functions in training neural network

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part II

Quantified Score

Hi-index	0.00

Visualization

Abstract

This work presents two novel approaches, backpropagation (BP) with magnified gradient function (MGFPROP) and deterministic weight modification (DWM), to speed up the convergence rate and improve the global convergence capability of the standard BP learning algorithm. The purpose of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function, while the main objective of DWM is to reduce the system error by changing the weights of a multilayered feedforward neural network in a deterministic way. Simulation results show that the performance of the above two approaches is better than BP and other modified BP algorithms for a number of learning problems. Moreover, the integration of the above two approaches forming a new algorithm called MDPROP, can further improve the performance of MGFPROP and DWM. From our simulation results, the MDPROP algorithm always outperforms BP and other modified BP algorithms in terms of convergence rate and global convergence capability.