The cascade-correlation learning architecture
Advances in neural information processing systems 2
A resource-allocating network for function interpolation
Neural Computation
Neural networks and the bias/variance dilemma
Neural Computation
Approximation and Estimation Bounds for Artificial Neural Networks
Machine Learning - Special issue on computational learning theory
Floating search methods in feature selection
Pattern Recognition Letters
Feature Selection for Knowledge Discovery and Data Mining
Feature Selection for Knowledge Discovery and Data Mining
Machine Learning
Predictive models for the breeder genetic algorithm i. continuous parameter optimization
Evolutionary Computation
Matching pursuits with time-frequency dictionaries
IEEE Transactions on Signal Processing
The cascade-correlation learning: a projection pursuit learning perspective
IEEE Transactions on Neural Networks
Use of bias term in projection pursuit learning improves approximation and convergence properties
IEEE Transactions on Neural Networks
IEEE Transactions on Neural Networks
Objective functions for training new hidden units in constructive neural networks
IEEE Transactions on Neural Networks
Exploring constructive cascade networks
IEEE Transactions on Neural Networks
Application of adaptive constructive neural networks to image compression
IEEE Transactions on Neural Networks
Orthogonal least squares learning algorithm for radial basis function networks
IEEE Transactions on Neural Networks
IEEE Transactions on Neural Networks
Regression modeling in back-propagation and projection pursuit learning
IEEE Transactions on Neural Networks
| Hi-index | 0.03 |
The selection of weights of the new hidden units for sequential feed-forward neural networks (FNNs) usually involves a non-linear optimization problem that cannot be solved analytically in the general case. A suboptimal solution is searched heuristically. Most models found in the literature choose the weights in the first layer that correspond to each hidden unit so that its associated output vector matches the previous residue as best as possible. The weights in the second layer can be either optimized (in a least-squares sense) or not. Several exceptions to the idea of matching the residue perform an (implicit or explicit) orthogonalization of the output vectors of the hidden units. In this case, the weights in the second layer are always optimized. An experimental study of the aforementioned approaches to select the weights for sequential FNNs is presented. Our results indicate that the orthogonalization of the output vectors of the hidden units outperforms the strategy of matching the residue, both for approximation and generalization purposes.