A novel method for constructing ensemble classifiers

Authors:
Chun-Xia Zhang;Jiang-She Zhang
Affiliations:
School of Science, Xi'an Jiaotong University, Xi'an Shaanxi, People's Republic of China 710049;School of Science, Xi'an Jiaotong University, Xi'an Shaanxi, People's Republic of China 710049
Venue:
Statistics and Computing
Year:
2009

Citing 15
Cited 1

Bagging predictors

Machine Learning
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
The Random Subspace Method for Constructing Decision Forests

IEEE Transactions on Pattern Analysis and Machine Intelligence
Approximate statistical tests for comparing supervised classification learning algorithms

Neural Computation
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
Random Forests

Machine Learning
Neural Network Ensembles

IEEE Transactions on Pattern Analysis and Machine Intelligence
Pruning Adaptive Boosting

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
An introduction to boosting and leveraging

Advanced lectures on machine learning
Combined 5 × 2 cv F Test for Comparing Supervised Classification Learning Algorithms

Neural Computation
Rotation Forest: A New Classifier Ensemble Method

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Comparison of Decision Tree Ensemble Creation Techniques

IEEE Transactions on Pattern Analysis and Machine Intelligence
Bundling classifiers by bagging trees

Computational Statistics & Data Analysis
Evolving hybrid ensembles of learning machines for better generalisation

Neurocomputing
Combining feature subsets in feature selection

MCS'05 Proceedings of the 6th international conference on Multiple Classifier Systems

DF-SVM: a decision forest constructed on artificially enlarged feature space by support vector machine

Artificial Intelligence Review

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a novel ensemble classifier generation method by integrating the ideas of bootstrap aggregation and Principal Component Analysis (PCA). To create each individual member of an ensemble classifier, PCA is applied to every out-of-bag sample and the computed coefficients of all principal components are stored, and then the principal components calculated on the corresponding bootstrap sample are taken as additional elements of the original feature set. A classifier is trained with the bootstrap sample and some features randomly selected from the new feature set. The final ensemble classifier is constructed by majority voting of the trained base classifiers. The results obtained by empirical experiments and statistical tests demonstrate that the proposed method performs better than or as well as several other ensemble methods on some benchmark data sets publicly available from the UCI repository. Furthermore, the diversity-accuracy patterns of the ensemble classifiers are investigated by kappa-error diagrams.