Using boosting to prune bagging ensembles

Authors:
Gonzalo Martínez-Muñoz;Alberto Suárez
Affiliations:
Escuela Politécnica Superior, Universidad Autónoma de Madrid, C/Francisco Tomás y Valiente, 11, Madrid E-28049, Spain;Escuela Politécnica Superior, Universidad Autónoma de Madrid, C/Francisco Tomás y Valiente, 11, Madrid E-28049, Spain
Venue:
Pattern Recognition Letters
Year:
2007

Citing 17
Cited 19

Bagging predictors

Machine Learning
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
MultiBoosting: A Technique for Combining Boosting and Wagging

Machine Learning
An approach to the automatic design of multiple classifier systems

Pattern Recognition Letters - Special issue on machine learning and data mining in pattern recognition
Soft Margins for AdaBoost

Machine Learning
Cost complexity-based pruning of ensemble classifiers

Knowledge and Information Systems
Random Forests

Machine Learning
Ensembling neural networks: many could be better than all

Artificial Intelligence
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Machine Learning
Knowledge Acquisition form Examples Vis Multiple Models

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
A decision-theoretic generalization of on-line learning and an application to boosting

EuroCOLT '95 Proceedings of the Second European Conference on Computational Learning Theory
Pruning Adaptive Boosting

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Clustering ensembles of neural network models

Neural Networks
Cost-conscious classifier ensembles

Pattern Recognition Letters
Switching class labels to generate classification ensembles

Pattern Recognition
Selective ensemble of decision trees

RSFDGrC'03 Proceedings of the 9th international conference on Rough sets, fuzzy sets, data mining, and granular computing
Using all data to generate decision tree ensembles

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews

Non-parametric bootstrap ensembles for detection of tumor lesions

Pattern Recognition Letters
Using Boosting to prune Double-Bagging ensembles

Computational Statistics & Data Analysis
Incremental construction of classifier and discriminant ensembles

Information Sciences: an International Journal
Pruning an ensemble of classifiers via reinforcement learning

Neurocomputing
Boosting One-Class Support Vector Machines for Multi-Class Classification

Applied Artificial Intelligence
A fast ensemble pruning algorithm based on pattern mining process

Data Mining and Knowledge Discovery
The impact of random samples in ensemble classifiers

Proceedings of the 2010 ACM Symposium on Applied Computing
Selection of decision stumps in bagging ensembles

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Sparse ensembles using weighted combination methods based on linear programming

Pattern Recognition
Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles

Neurocomputing
Margin distribution based bagging pruning

Neurocomputing
A double pruning algorithm for classification ensembles

MCS'10 Proceedings of the 9th international conference on Multiple Classifier Systems
Ensemble pruning for text categorization based on data partitioning

AIRS'11 Proceedings of the 7th Asia conference on Information Retrieval Technology
Expert pruning based on genetic algorithm in regression problems

ACIIDS'12 Proceedings of the 4th Asian conference on Intelligent Information and Database Systems - Volume Part III
Ensemble approaches for regression: A survey

ACM Computing Surveys (CSUR)
Letters: Dynamic classifier ensemble using classification confidence

Neurocomputing
A competitive ensemble pruning approach based on cross-validation technique

Knowledge-Based Systems
An effective ensemble pruning algorithm based on frequent patterns

Knowledge-Based Systems
DF-SVM: a decision forest constructed on artificially enlarged feature space by support vector machine

Artificial Intelligence Review

Quantified Score

Hi-index	0.10

Visualization

Abstract

Boosting is used to determine the order in which classifiers are aggregated in a bagging ensemble. Early stopping in the aggregation of the classifiers in the ordered bagging ensemble allows the identification of subensembles that require less memory for storage, classify faster and can improve the generalization accuracy of the original bagging ensemble. In all the classification problems investigated pruned ensembles with 20% of the original classifiers show statistically significant improvements over bagging. In problems where boosting is superior to bagging, these improvements are not sufficient to reach the accuracy of the corresponding boosting ensembles. However, ensemble pruning preserves the performance of bagging in noisy classification tasks, where boosting often has larger generalization errors. Therefore, pruned bagging should generally be preferred to complete bagging and, if no information about the level of noise is available, it is a robust alternative to AdaBoost.