Novel Training Algorithm Based on Quadratic Optimisation Using Neural Networks

Authors:
Ganesh Arulampalam;Abdesselam Bouzerdoum
Affiliations:
-;-
Venue:
IWANN '01 Proceedings of the 6th International Work-Conference on Artificial and Natural Neural Networks: Connectionist Models of Neurons, Learning Processes and Artificial Intelligence-Part I
Year:
2001

Citing 2
Cited 0

First- and second-order methods for learning: between steepest descent and Newton's method

Neural Computation
Neural Networks: A Comprehensive Foundation

Neural Networks: A Comprehensive Foundation

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we present a novel algorithm for training feedforward neural networks based on the use of recurrent neural networks for bound constrained quadratic optimisation. Instead of trying to invert the Hessian matrix or its approximation, as done in other second-order algorithms, a recurrent equation that emulates a recurrent neural network determines the optimal weight update. The development of this algorithm is presented, along with its performance under ideal conditions as well as results from training multilayer perceptrons. The results show that the algorithm is capable of achieving results with less errors than other methods for a variety of problems.