A Fast Heuristic Global Learning Algorithm for Multilayer Neural Networks

Authors:
Siu-Yeung Cho;Tommy W. S. Chow
Affiliations:
Department of Electronic Engineering, City University of Hong Kong, Hong Kong. e-mail: sycho@ee.cityu.edu.hk;Department of Electronic Engineering, City University of Hong Kong, Hong Kong. e-mail: sycho@ee.cityu.edu.hk
Venue:
Neural Processing Letters
Year:
1999

Citing 6
Cited 0

Simulated annealing and Boltzmann machines: a stochastic approach to combinatorial optimization and neural computing

Simulated annealing and Boltzmann machines: a stochastic approach to combinatorial optimization and neural computing
Global optimization

Global optimization
On the Problem of Local Minima in Backpropagation

IEEE Transactions on Pattern Analysis and Machine Intelligence
Integral global optimization of constrained problems in functional spaces with discontinuous penalty functions

Recent advances in global optimization
Global Optimization for Neural Network Training

Computer - Special issue: neural computing: companion issue to Spring 1996 IEEE Computational Science & Engineering
Training neural nets with the reactive tabu search

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

This paper presents a novel Heuristic Global Learning (HER-GBL) algorithm for multilayer neural networks. The algorithm is based upon the least squares method to maintain the fast convergence speed, and the penalized optimization to solve the problem of local minima. The penalty term, defined as a Gaussian-type function of the weight, is to provide an uphill force to escape from local minima. As a result, the training performance is dramatically improved. The proposed HER-GBL algorithm yields excellent results in terms of convergence speed, avoidance of local minima and quality of solution.