Improved computation for Levenberg-Marquardt training

Authors:
Bogdan M. Wilamowski;Hao Yu
Affiliations:
Department of Electrical and Computer Engineering, Auburn University, Auburn, AL;Department of Electrical and Computer Engineering, Auburn University, Auburn, AL
Venue:
IEEE Transactions on Neural Networks
Year:
2010

Citing 11
Cited 4

Efficient and reliable training of neural networks

HSI'09 Proceedings of the 2nd conference on Human System Interactions
Two highly efficient second-order algorithms for training feedforward networks

IEEE Transactions on Neural Networks
Parallel sequential minimal optimization for the training of support vector machines

IEEE Transactions on Neural Networks
Accurate and Fast Off and Online Fuzzy ARTMAP-Based Image Classification With Application to Genetic Abnormality Diagnosis

IEEE Transactions on Neural Networks
Weighted Piecewise LDA for Solving the Small Sample Size Problem in Face Verification

IEEE Transactions on Neural Networks
Discrete-Time Adaptive Backstepping Nonlinear Control via High-Order Neural Networks

IEEE Transactions on Neural Networks
A New Jacobian Matrix for Optimal Learning of Single-Layer Neural Networks

IEEE Transactions on Neural Networks
Training Two-Layered Feedforward Networks With Variable Projection Method

IEEE Transactions on Neural Networks
Optimized Approximation Algorithm in Neural Networks Without Overfitting

IEEE Transactions on Neural Networks
A Constrained Optimization Approach to Preserving Prior Knowledge During Incremental Training

IEEE Transactions on Neural Networks
Multilayer Potts Perceptrons With Levenberg–Marquardt Learning

IEEE Transactions on Neural Networks

Neural network learning without backpropagation

IEEE Transactions on Neural Networks
Surface recognition improvement in 3D medical laser scanner using Levenberg-Marquardt method

Signal Processing
Evolutionary algorithm assisted by surrogate model in the framework of ordinal optimization and optimal computing budget allocation

Information Sciences: an International Journal
Original Articles: SOM neural network design - A new Simulink library based approach targeting FPGA implementation

Mathematics and Computers in Simulation

Quantified Score

Hi-index	0.00

Visualization

Abstract

The improved computation presented in this paper is aimed to optimize the neural networks learning process using Levenberg-Marquardt (LM) algorithm. Quasi-Hessian matrix and gradient vector are computed directly, without Jacobian matrix multiplication and storage. The memory limitation problem for LM training is solved. Considering the symmetry of quasi-Hessian matrix, only elements in its upper/lower triangular array need to be calculated. Therefore, training speed is improved significantly, not only because of the smaller array stored in memory, but also the reduced operations in quasi-Hessian matrix calculation. The improved memory and time efficiencies are especially true for large sized patterns training.