A node pruning algorithm based on optimal brain surgeon for feedforward neural networks

Authors:
Jinhua Xu;Daniel W. C. Ho
Affiliations:
Department of Computer Sciences, East China Normal University;Department of Mathematics, City University of Hong Kong
Venue:
ISNN'06 Proceedings of the Third international conference on Advances in Neural Networks - Volume Part I
Year:
2006

Citing 5
Cited 0

Regularization theory and neural networks architectures

Neural Computation
Second Order Derivatives for Network Pruning: Optimal Brain Surgeon

Advances in Neural Information Processing Systems 5, [NIPS Conference]
New training strategies for constructive neural networks with application to regression problems

Neural Networks
An iterative pruning algorithm for feedforward neural networks

IEEE Transactions on Neural Networks
Constructive algorithms for structure learning in feedforward neural networks for regression problems

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, a node pruning algorithm based on optimal brain surgeon is proposed for feedforward neural networks. First, the neural network is trained to an acceptable solution using the standard training algorithm. After the training process, the orthogonal factorization is applied to the output of the nodes in the same hidden layer to identify and prune the dependant nodes. Then, a unit-based optimal brain surgeon(UB-OBS) pruning algorithm is proposed to prune the insensitive hidden units to further reduce the size of the neural network, and no retraining is needed. Simulations are presented to demonstrate the effectiveness of the proposed approach.