Optimal brain surgeon for general dynamic neural networks

Authors:
Christian Endisch;Christoph Hackl;Dierk Schröder
Affiliations:
Institute for Electrical Drive Systems, Technical University of Munich, München, Germany;Institute for Electrical Drive Systems, Technical University of Munich, München, Germany;Institute for Electrical Drive Systems, Technical University of Munich, München, Germany
Venue:
EPIA'07 Proceedings of the aritficial intelligence 13th Portuguese conference on Progress in artificial intelligence
Year:
2007

Citing 5
Cited 2

Optimal brain damage

Advances in neural information processing systems 2
Training general dynamic neural networks

Training general dynamic neural networks
A learning algorithm for continually running fully recurrent neural networks

Neural Computation
Efficient training of neural nets for nonlinear adaptive filteringusing a recursive Levenberg-Marquardt algorithm

IEEE Transactions on Signal Processing
Backpropagation Algorithms for a Broad Class of Dynamic Networks

IEEE Transactions on Neural Networks

Comments on "Backpropagation algorithms for a broad class of dynamic networks"

IEEE Transactions on Neural Networks
Levenberg-Marquardt-based OBS algorithm using adaptive pruning interval for system identification with dynamic neural networks

SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a pruning algorithm based on optimal brain surgeon (OBS) for general dynamic neural networks (GDNN). The pruning algorithm uses Hessian information and considers the order of time delay for saliency calculation. In GDNNs all layers have feedback connections with time delays to the same and to all other layers. The parameters are trained with the Levenberg-Marquardt (LM) algorithm. Therefore the Jacobian matrix is required. The Jacobian is calculated by real time recurrent learning (RTRL). As both LM and OBS need Hessian information, a rational implementation is suggested.