A network pruning algorithm for combined function and derivative approximation

Authors:
Arjpolson Pukrittayakamee;Martin Hagan;Lionel Raff;Satish Bukkapatnam;Ranga Komanduri
Affiliations:
School of Electrical and Computer Engineering, Oklahoma State University, Stillwater, OK;School of Electrical and Computer Engineering, Oklahoma State University, Stillwater, OK;Chemistry Department, Oklahoma State University, Stillwater, OK;School of Industrial Engineering and Management, Oklahoma State University, Stillwater, OK;School of Mechanical and Aerospace Engineering, Oklahoma State University, Stillwater, OK
Venue:
IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Year:
2009

Citing 6
Cited 0

Bayesian interpolation

Neural Computation
Neural network design

Neural network design
.NET Test Automation Recipes: A Problem-Solution Approach

.NET Test Automation Recipes: A Problem-Solution Approach
A new pruning heuristic based on variance analysis of sensitivity information

IEEE Transactions on Neural Networks
Smooth function approximation using neural networks

IEEE Transactions on Neural Networks
A node pruning algorithm based on a Fourier amplitude sensitivity test method

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes newly discovered types of overfitting that occur when sim ultaneously fitting a function and its first derivatives with multilayer feedforward neural networks. We analyze the overfitting and demonstrate how it develops. These types of overfitting occur over very narrow regions in the input space, thus a validation set is not helpful in detecting them. A new pruning algorithm is proposed to eliminate these types of overfitting. Simulation results show that the pruning algorithm successfully eliminates the overfitting, produces smooth responses and provides excellent generalization capabilities. The proposed pruning algorithm can be used with any single-output, two-layer network, which uses a hyperbolic tangent transfer function in the hidden layer.