Parameter by Parameter Algorithm for Multilayer Perceptrons

  • Authors:
  • Yanlai Li;David Zhang;Kuanquan Wang

  • Affiliations:
  • Biocomputing Research Centre, School of Computer Science and Technology, Harbin Institute of Technology (HIT), Harbin, China;Biometric Research Centre (UGC/CRC)/Department of Computing, Hong Kong Polytechnic University, Kowloon, Hong Kong;Biocomputing Research Centre, School of Computer Science and Technology, Harbin Institute of Technology (HIT), Harbin, China

  • Venue:
  • Neural Processing Letters
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper presents a parameter by parameter (PBP) algorithm for speeding up the training of multilayer perceptrons (MLP). This new algorithm uses an approach similar to that of the layer by layer (LBL) algorithm, taking into account the input errors of the output layer and hidden layer. The proposed PBP algorithm, however, is not burdened by the need to calculate the gradient of the error function. In each iteration step, the weights or thresholds can be optimized directly one by one with other variables fixed. Four classes of solution equations for parameters of networks are deducted. The effectiveness of the PBP algorithm is demonstrated using two benchmarks. In comparisons with the BP algorithm with momentum (BPM) and the conventional LBL algorithms, PBP obtains faster convergences and better simulation performances.