Comparison of Bagging and Boosting Algorithms on Sample and Feature Weighting

Authors:
Satoshi Shirai;Mineichi Kudo;Atsuyoshi Nakamura
Affiliations:
Division of Computer Science Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan 060-0814;Division of Computer Science Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan 060-0814;Division of Computer Science Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan 060-0814
Venue:
MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
Year:
2009

Citing 12
Cited 2

C4.5: programs for machine learning

C4.5: programs for machine learning
Bagging predictors

Machine Learning
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Approximate statistical tests for comparing supervised classification learning algorithms

Neural Computation
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
Random Forests

Machine Learning
Ensemble Methods in Machine Learning

MCS '00 Proceedings of the First International Workshop on Multiple Classifier Systems
Global Data Analysis and the Fragmentation Problem in Decision Tree Induction

ECML '97 Proceedings of the 9th European Conference on Machine Learning
Combining Pattern Classifiers: Methods and Algorithms

Combining Pattern Classifiers: Methods and Algorithms
Combined 5 × 2 cv F Test for Comparing Supervised Classification Learning Algorithms

Neural Computation
Extremely randomized trees

Machine Learning
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)

Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)

Ensemble based sensing anomaly detection in wireless sensor networks

Expert Systems with Applications: An International Journal
A noise-detection based AdaBoost algorithm for mislabeled data

Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

We compared boosting with bagging in different strengths of learning algorithms for improving the performance of the set of classifiers to be fused. Our experimental results showed that boosting worked much with weak algorithms and bagging, especially feature-based bagging, worked much with strong algorithms. On the basis of these observations we developed a mixed fusion method in which randomly chosen features are used with a standard boosting method. As a result, it was confirmed that the proposed fusion method worked well regardless of learning algorithms.