Applied regression analysis and other multivariable methods
Applied regression analysis and other multivariable methods
Original Contribution: Stacked generalization
Neural Networks
Hierarchical mixtures of experts and the EM algorithm
Neural Computation
Machine Learning
Optimal linear combinations of neural networks
Neural Networks
Neural Networks for Pattern Recognition
Neural Networks for Pattern Recognition
Ensembling neural networks: many could be better than all
Artificial Intelligence
Introduction to the Theory of Neural Computation
Introduction to the Theory of Neural Computation
Neural Computation
Adaptive mixtures of local experts
Neural Computation
Hi-index | 0.01 |
Combining multiple neural networks appears to be a very promising approach in improving neural network generalisation since it is very difficult, if not impossible, to develop a perfect single neural network. In the building of an aggregated neural network model, a number of individual networks are developed from different data sets and/or different training algorithms. In this paper, individual networks are developed from bootstrap re-samples of the original training and testing data sets. Instead of combining all the developed networks, this paper proposes two selective combination techniques: forward selection and backward elimination. These two techniques essentially combine those individual networks that, when combined, can significantly improve model generalisation. In forward selection, individual networks are gradually added into the aggregated network until the aggregated network error on the original training and testing data sets cannot be further reduced. In backward elimination, all the individual networks are initially aggregated and some of the individual networks are then gradually eliminated until the aggregated network error on the original training and testing data sets cannot be further reduced. The proposed techniques are applied to dynamic nonlinear process modelling and classification of diabetes database. Application results demonstrate that the proposed techniques can significantly improve model generalisation and perform better than aggregating all the individual networks and the heuristic selective combination method where networks with better performance on the training and testing data are selected.