An integrated growing-pruning method for feedforward network training

Authors:
Pramod L. Narasimha;Walter H. Delashmit;Michael T. Manry;Jiang Li;Francisco Maldonado
Affiliations:
Department of Electrical Engineering, University of Texas at Arlington, Arlington, TX 76013, USA;Lockheed Martin, Missiles and Fire Control Dallas, Grand Prairie, TX 75051, USA;Department of Electrical Engineering, University of Texas at Arlington, Arlington, TX 76013, USA;Electrical and Computer Engineering, Old Dominion University, Norfolk, VA 23529, USA;Williams Pyro, Inc., 200 Greenleaf Street, Fort Worth, TX 76107, USA
Venue:
Neurocomputing
Year:
2008

Citing 15
Cited 10

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations
Back-propagation algorithm which varies the number of hidden units

Neural Networks
The cascade-correlation learning architecture

Advances in neural information processing systems 2
Modeling with constructive backpropagation

Neural Networks
Generalization in Neural Networks and Machine Learning

Generalization in Neural Networks and Machine Learning
Feedforward Neural Network Construction Using Cross Validation

Neural Computation
A fast new algorithm for training feedforward neural networks

IEEE Transactions on Signal Processing
Kernel orthonormalization in radial basis function neural networks

IEEE Transactions on Neural Networks
A formal selection and pruning algorithm for feedforward artificial neural network optimization

IEEE Transactions on Neural Networks
Parallel growing and training of neural networks using output parallelism

IEEE Transactions on Neural Networks
Orthogonal least squares learning algorithm for radial basis function networks

IEEE Transactions on Neural Networks
A simple method to derive bounds on the size and to train multilayer neural networks

IEEE Transactions on Neural Networks
A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation

IEEE Transactions on Neural Networks
Pruning algorithms-a survey

IEEE Transactions on Neural Networks
Genetic evolution of the topology and weight distribution of neural networks

IEEE Transactions on Neural Networks

A novel pruning algorithm for self-organizing neural network

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
A self-organizing neural network using fast training and pruning

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
A methodology for developing nonlinear models by feedforward neural networks

ICANNGA'09 Proceedings of the 9th international conference on Adaptive and natural computing algorithms
POFGEC: growing neural network of classifying potential function generators

International Journal of Knowledge Engineering and Soft Data Paradigms
An elitist non-dominated sorting genetic algorithm enhanced with a neural network applied to the multi-objective optimization of a polysiloxane synthesis process

Engineering Applications of Artificial Intelligence
A Novel Pruning Algorithm for Optimizing Feedforward Neural Network of Classification Problems

Neural Processing Letters
A structure optimisation algorithm for feedforward neural network construction

Neurocomputing
Efficient self-organizing multilayer neural network for nonlinear system modeling

Neural Networks
Particle swarm optimization for construction of neural network-based prediction intervals

Neurocomputing
Neural networks letter: Neural architecture design based on extreme learning machine

Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

In order to facilitate complexity optimization in feedforward networks, several algorithms are developed that combine growing and pruning. First, a growing scheme is presented which iteratively adds new hidden units to full-trained networks. Then, a non-heuristic one-pass pruning technique is presented, which utilizes orthogonal least squares. Based upon pruning, a one-pass approach is developed for generating the validation error versus network size curve. A combined approach is described in which networks are continually pruned during the growing process. As a result, the hidden units are ordered according to their usefulness, and the least useful units are eliminated. Examples show that networks designed using the combined method have less training and validation error than growing or pruning alone. The combined method exhibits reduced sensitivity to the initial weights and generates an almost monotonic error versus network size curve. It is shown to perform better than two well-known growing methods-constructive backpropagation and cascade correlation.