Robust training of feedforward neural networks using combined online/batch quasi-newton techniques

Authors:
Hiroshi Ninomiya
Affiliations:
Department of Information Science, Shonan Institute of Technology, Fujisawa, Kanagawa, Japan
Venue:
ICANN'12 Proceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part II
Year:
2012

Citing 5
Cited 0

Minimizing multimodal functions of continuous variables with the “simulated annealing” algorithm: Corrigenda for this article is available here

ACM Transactions on Mathematical Software (TOMS)
Recursive stochastic algorithms for global optimization in Rd

SIAM Journal on Control and Optimization
On Langevin updating in multilayer perceptrons

Neural Computation
New training strategies for constructive neural networks with application to regression problems

Neural Networks
Objective functions for training new hidden units in constructive neural networks

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes a robust training algorithm based on quasi-Newton process in which online and batch error functions are combined by a weighting coefficient parameter. The parameter is adjusted to ensure that the algorithm gradually changes from online to batch. Furthermore, an analogy between this algorithm and Langevin one is considered. Langevin algorithm is a gradient-based continuous optimization method incorporating Simulated Annealing concept. Neural network training is presented to demonstrate the validity of combined algorithm. The algorithm achieves more robust training and accurate generalization results than other quasi-Newton based training algorithms.