Ensemble algorithms for feature selection

Authors:
Jeremy D. Rogers;Steve R. Gunn
Affiliations:
Image, Speech and Intelligent Systems Research Group, School of Electronics and Computer Science, University of Southampton, U.K.;Image, Speech and Intelligent Systems Research Group, School of Electronics and Computer Science, University of Southampton, U.K.
Venue:
Proceedings of the First international conference on Deterministic and Statistical Methods in Machine Learning
Year:
2004

Citing 6
Cited 1

Bagging predictors

Machine Learning
Feature selection for ensembles

AAAI '99/IAAI '99 Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
Random Forests

Machine Learning
Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Nearest Neighbors in Random Subspaces

SSPR '98/SPR '98 Proceedings of the Joint IAPR International Workshops on Advances in Pattern Recognition

Identifying feature relevance using a random forest

SLSFS'05 Proceedings of the 2005 international conference on Subspace, Latent Structure and Feature Selection

Quantified Score

Hi-index	0.00

Visualization

Abstract

Many feature selection algorithms are limited in that they attempt to identify relevant feature subsets by examining the features individually. This paper introduces a technique for determining feature relevance using the average information gain achieved during the construction of decision tree ensembles. The technique introduces a node complexity measure and a statistical method for updating the feature sampling distribution based upon confidence intervals to control the rate of convergence. A feature selection threshold is also derived, using the expected performance of an irrelevant feature. Experiments demonstrate the potential of these methods and illustrate the need for both feature weighting and selection.