A fusion of stacking with dynamic integration

Authors:
Niall Rooney;David Patterson
Affiliations:
Northern Ireland Knowledge Engineering Laboratory, Faculty of Engineering, University of Ulster, Jordanstown, Newtownabbey, UK;Northern Ireland Knowledge Engineering Laboratory, Faculty of Engineering, University of Ulster, Jordanstown, Newtownabbey, UK
Venue:
IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Year:
2007

Citing 11
Cited 0

Original Contribution: Stacked generalization

Neural Networks
Stacked regressions

Machine Learning
Bagging predictors

Machine Learning
Locally Weighted Learning

Artificial Intelligence Review - Special issue on lazy learning
The Random Subspace Method for Constructing Decision Forests

IEEE Transactions on Pattern Analysis and Machine Intelligence
Data mining: practical machine learning tools and techniques with Java implementations

Data mining: practical machine learning tools and techniques with Java implementations
Nearest Neighbors in Random Subspaces

SSPR '98/SPR '98 Proceedings of the Joint IAPR International Workshops on Advances in Pattern Recognition
Exploring the Parameter State Space of Stacking

ICDM '02 Proceedings of the 2002 IEEE International Conference on Data Mining
Is Combining Classifiers with Stacking Better than Selecting the Best One?

Machine Learning
Sequential genetic search for ensemble feature selection

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Extraction of rules from artificial neural networks for nonlinear regression

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we present a novel method that fuses the ensemble meta-techniques of Stacking and Dynamic Integration (DI) for regression problems, without adding any major computational overhead. The intention of the technique is to benefit from the varying performance of Stacking and DI for different data sets, in order to provide a more robust technique. We detail an empirical analysis of the technique referred to as weighted Meta - Combiner (wMetaComb) and compare its performance to Stacking and the DI technique of Dynamic Weighting with Selection. The empirical analysis consisted of four sets of experiments where each experiment recorded the cross-fold evaluation of each technique for a large number of diverse data sets, where each base model is created using random feature selection and the same base learning algorithm. Each experiment differed in terms of the latter base learning algorithm used. We demonstrate that for each evaluation, wMeta-Comb was able to outperform DI and Stacking for each experiment and as such fuses the two underlying mechanisms successfully.