Speeding up the wrapper feature subset selection in regression by mutual information relevance and redundancy analysis

Authors:
Gert Van Dijck;Marc M. Van Hulle
Affiliations:
Computational Neuroscience Research Group, Laboratorium voor Neuro-en Psychofysiologie, K.U. Leuven, Leuven, Belgium;Computational Neuroscience Research Group, Laboratorium voor Neuro-en Psychofysiologie, K.U. Leuven, Leuven, Belgium
Venue:
ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part I
Year:
2006

Citing 9
Cited 9

Elements of information theory

Elements of information theory
Floating search methods in feature selection

Pattern Recognition Letters
Genetic algorithms + data structures = evolution programs (3rd ed.)

Genetic algorithms + data structures = evolution programs (3rd ed.)
Wrappers for feature subset selection

Artificial Intelligence - Special issue on relevance
Surrogate time series

Physica D
An introduction to variable and feature selection

The Journal of Machine Learning Research
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)
CAIM Discretization Algorithm

IEEE Transactions on Knowledge and Data Engineering
A Branch and Bound Algorithm for Feature Subset Selection

IEEE Transactions on Computers

Resampling methods for parameter-free and robust feature selection with mutual information

Neurocomputing
Agglomerative independent variable group analysis

Neurocomputing
Speeding Up Feature Subset Selection Through Mutual Information Relevance Filtering

PKDD 2007 Proceedings of the 11th European conference on Principles and Practice of Knowledge Discovery in Databases
Advances in Feature Selection with Mutual Information

Similarity-Based Clustering
Posterior probability profiles for the automated assessment of the recovery of stroke patients

AAAI'07 Proceedings of the 22nd national conference on Artificial intelligence - Volume 1
Normalized mutual information feature selection

IEEE Transactions on Neural Networks
Weighted mutual information for feature selection

ICANN'11 Proceedings of the 21st international conference on Artificial neural networks - Volume Part II
Neural networks letter: Is mutual information adequate for feature selection in regression?

Neural Networks
New automated power quality recognition system for online/offline monitoring

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

A hybrid filter/wrapper feature subset selection algorithm for regression is proposed. First, features are filtered by means of a relevance and redundancy filter using mutual information between regression and target variables. We introduce permutation tests to find statistically significant relevant and redundant features. Second, a wrapper searches for good candidate feature subsets by taking the regression model into account. The advantage of a hybrid approach is threefold. First, the filter provides interesting features independently from the regression model and, hence, allows for an easier interpretation. Secondly, because the filter part is computationally less expensive, the global algorithm will faster provide good candidate subsets compared to a stand-alone wrapper approach. Finally, the wrapper takes the bias of the regression model into account, because the regression model guides the search for optimal features. Results are shown for the ‘Boston housing’ and ‘orange juice’ benchmarks based on the multilayer perceptron regression model.