Removal of hidden neurons in multilayer perceptrons by orthogonal projection and weight crosswise propagation

Authors:
Xun Liang
Affiliations:
Institute of Computer Science and Technology, Peking University, 100871, Beijing, China and Department of Economics and Operations Research, Stanford University, Stanford, CA, 95035, USA
Venue:
Neural Computing and Applications
Year:
2006

Citing 0
Cited 2

Neural network architecture selection: can function complexity help?

Neural Processing Letters
Variance sensitivity analysis of parameters for pruning of a multilayer perceptron: application to a sawmill supply chain simulation model

Advances in Artificial Neural Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

A new method of pruning away hidden neurons in neural networks is presented in this paper. The hidden neuron is removed by analyzing the orthogonal projection correlations among the outputs of other hidden neurons. The method guarantees the least loss of weight information in terms of orthogonal projection. The remaining weights and thresholds are updated based on the weight crosswise propagation. A practical technique for penalizing the superfluous hidden neurons is explored. Retraining is needed after pruning. Extensive experiments are conducted, and the results demonstrate that the method gives better initial points for retraining and retraining costs less epochs.