Two highly efficient second-order algorithms for training feedforward networks

Authors:
N. Ampazis;S. J. Perantonis
Affiliations:
Inst. of Informatics & Telecommun., Nat. Center for Sci. Res. "DEMOKRITOS", Athens;-
Venue:
IEEE Transactions on Neural Networks
Year:
2002

Citing 0
Cited 16

A hierarchical learning network for face detection with in-plane rotation

Neurocomputing
Two-stage mixed discrete-continuous identification of radial basis function (RBF) neural models for nonlinear systems

IEEE Transactions on Circuits and Systems Part I: Regular Papers
Template-free symbolic performance modeling of analog circuits via canonical-form functions and genetic programming

IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Globally reliable variation-aware sizing of analog integrated circuits via response surfaces and structural homotopy

IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
A modified gradient-based neuro-fuzzy learning algorithm and its convergence

Information Sciences: an International Journal
Improved computation for Levenberg-Marquardt training

IEEE Transactions on Neural Networks
Analysis of an evolutionary RBFN design algorithm, CO2RBFN, for imbalanced data sets

Pattern Recognition Letters
Neural network learning without backpropagation

IEEE Transactions on Neural Networks
Training neural networks by rational weight functions

AICI'11 Proceedings of the Third international conference on Artificial intelligence and computational intelligence - Volume Part III
Rotation invariant face detection using convolutional neural networks

ICONIP'06 Proceedings of the 13th international conference on Neural Information Processing - Volume Part II
Deterministic global optimization with a neighbourhood determination algorithm based on neural networks

ISNN'05 Proceedings of the Second international conference on Advances in Neural Networks - Volume Part I
A global-local optimization approach to parameter estimation of RBF-type models

Information Sciences: an International Journal
Design of second order neural networks as dynamical control systems that aim to minimize nonconvex scalar functions

Neurocomputing
Hierarchical analog circuit reliability analysis using multivariate nonlinear regression and active learning sample selection

DATE '12 Proceedings of the Conference on Design, Automation and Test in Europe
Parallel computation of a new data driven algorithm for training neural networks

ISNN'13 Proceedings of the 10th international conference on Advances in Neural Networks - Volume Part I
Batch gradient method with smoothing L1/ 2 regularization for training of feedforward neural networks

Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present two highly efficient second-order algorithms for the training of multilayer feedforward neural networks. The algorithms are based on iterations of the form employed in the Levenberg-Marquardt (LM) method for nonlinear least squares problems with the inclusion of an additional adaptive momentum term arising from the formulation of the training task as a constrained optimization problem. Their implementation requires minimal additional computations compared to a standard LM iteration. Simulations of large scale classical neural-network benchmarks are presented which reveal the power of the two methods to obtain solutions in difficult problems, whereas other standard second-order techniques (including LM) fail to converge.