Observations on boosting feature selection

Authors:
D. B. Redpath;K. Lebart
Affiliations:
ECE, School of EPS, Heriot-Watt University, Edinburgh, UK;ECE, School of EPS, Heriot-Watt University, Edinburgh, UK
Venue:
MCS'05 Proceedings of the 6th international conference on Multiple Classifier Systems
Year:
2005

Citing 10
Cited 0

A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
The Random Subspace Method for Constructing Decision Forests

IEEE Transactions on Pattern Analysis and Machine Intelligence
Improved Boosting Algorithms Using Confidence-rated Predictions

Machine Learning - The Eleventh Annual Conference on computational Learning Theory
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy

Machine Learning
Diversity versus Quality in Classification Ensembles Based on Feature Selection

ECML '00 Proceedings of the 11th European Conference on Machine Learning
Input Decimation Ensembles: Decorrelation through Dimensionality Reduction

MCS '01 Proceedings of the Second International Workshop on Multiple Classifier Systems
Bagging and Boosting for the Nearest Mean Classifier: Effects of Sample Size on Diversity and Accuracy

MCS '02 Proceedings of the Third International Workshop on Multiple Classifier Systems
Maximizing the Margin with Boosting

COLT '02 Proceedings of the 15th Annual Conference on Computational Learning Theory
Feature selection algorithms for the generation of multiple classifier systems and their application to handwritten word recognition

Pattern Recognition Letters
Bagging, boosting, and C4.S

AAAI'96 Proceedings of the thirteenth national conference on Artificial intelligence - Volume 1

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a study of the Boosting Feature Selection (BFS) algorithm [1], a method which incorporates feature selection into Adaboost. Such an algorithm is interesting as it combines the methods studied by Boosting and ensemble feature selection researchers. Observations are made on generalisation, weighted error and error diversity to compare the algorithms performance to Adaboost while using a nearest mean base learner. Ensemble feature prominence is proposed as a stop criterion for ensemble construction. Its quality assessed using the former performance measures. BFS is found to compete with Adaboost in terms of performance, despite the reduced feature description for each base classifer. This is explained using weighted error and error diversity. Results show the proposed stop criterion to be useful for trading ensemble performance and complexity.