Feature-subspace aggregating: ensembles for stable and unstable learners

Authors:
Kai Ming Ting;Jonathan R. Wells;Swee Chuan Tan;Shyh Wei Teng;Geoffrey I. Webb
Affiliations:
Gippsland School of Information Technology, Monash University, Vic, Australia 3842;Gippsland School of Information Technology, Monash University, Vic, Australia 3842;Gippsland School of Information Technology, Monash University, Vic, Australia 3842;Gippsland School of Information Technology, Monash University, Vic, Australia 3842;Clayton School of Information Technology, Monash University, Vic, Australia 3800
Venue:
Machine Learning
Year:
2011

Citing 0
Cited 4

Improving classifications for cardiac autonomic neuropathy using multi-level ensemble classifiers and feature selection based on random forest

AusDM '12 Proceedings of the Tenth Australasian Data Mining Conference - Volume 134
An investigation into the application of ensemble learning for entailment classification

Information Processing and Management: an International Journal
A survey of multiple classifier systems as hybrid systems

Information Fusion
Integrating global and local application of random subspace ensemble

Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper introduces a new ensemble approach, Feature-Subspace Aggregating (Feating), which builds local models instead of global models. Feating is a generic ensemble approach that can enhance the predictive performance of both stable and unstable learners. In contrast, most existing ensemble approaches can improve the predictive performance of unstable learners only. Our analysis shows that the new approach reduces the execution time to generate a model in an ensemble through an increased level of localisation in Feating. Our empirical evaluation shows that Feating performs significantly better than Boosting, Random Subspace and Bagging in terms of predictive accuracy, when a stable learner SVM is used as the base learner. The speed up achieved by Feating makes feasible SVM ensembles that would otherwise be infeasible for large data sets. When SVM is the preferred base learner, we show that Feating SVM performs better than Boosting decision trees and Random Forests. We further demonstrate that Feating also substantially reduces the error of another stable learner, k-nearest neighbour, and an unstable learner, decision tree.