A fast and numerically robust neural network training algorithm

  • Authors:

  • Youmin Zhang

  • Affiliations:

  • Department of Computer Science and Engineering, Aalborg University Esbjerg, Esbjerg, Denmark

  • Venue:
  • ICAISC'06 Proceedings of the 8th international conference on Artificial Intelligence and Soft Computing
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

A fast and numerically robust algorithm for training feedforward neural networks (FNNs) using a recursive prediction error (RPE) method, along with an adaptively-adjustable time-varying forgetting factor technique, is presented first. Then a U-D factorization-based RPE (UD-RPE) algorithm is proposed to further improve the training rate and accuracy of the FNNs. In comparison with the backpropagation (BP) and existing RPE based training algorithms, the proposed UD-RPE can provide substantially better results in fewer iterations with fewer hidden nodes and improve significantly convergence rate and numerical stability. In addition, it is less sensitive to start-up parameters, such as initial weights and covariance matrix. It has also good model prediction ability and need less training time. The effectiveness of the proposed algorithm is demonstrated via two nonlinear systems modeling and identification examples.