Input Variable Selection: Mutual Information and Linear Mixing Measures

Authors:
Thomas Trappenberg;Jie Ouyang;Andrew Back
Affiliations:
-;-;IEEE
Venue:
IEEE Transactions on Knowledge and Data Engineering
Year:
2006

Citing 8
Cited 4

Learning hard concepts through constructive induction: framework and rationale

Computational Intelligence
Wrappers for feature subset selection

Artificial Intelligence - Special issue on relevance
Understanding the Crucial Role of AttributeInteraction in Data Mining

Artificial Intelligence Review
An introduction to variable and feature selection

The Journal of Machine Learning Research
Identification of attribute interactions and generation of globally relevant continuous features in machine learning

Identification of attribute interactions and generation of globally relevant continuous features in machine learning
Estimation of the information by an adaptive partitioning of the observation space

IEEE Transactions on Information Theory
Selecting inputs for modeling using normalized higher order statistics and independent component analysis

IEEE Transactions on Neural Networks
Estimating optimal feature subsets using efficient estimation of high-dimensional mutual information

IEEE Transactions on Neural Networks

Evaluation and classification of otoneurological data with new data analysis methods based on machine learning

Information Sciences: an International Journal
A minimax mutual information scheme for supervised feature extraction and its application to EEG-based brain-computer interfacing

EURASIP Journal on Advances in Signal Processing
Feature selection based on run covering

PSIVT'06 Proceedings of the First Pacific Rim conference on Advances in Image and Video Technology
A scatter method for data and variable importance evaluation

Integrated Computer-Aided Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

Determining the most appropriate inputs to a model has a significant impact on the performance of the model and associated algorithms for classification, prediction, and data analysis. Previously, we proposed an algorithm ICAIVS which utilizes independent component analysis (ICA) as a preprocessing stage to overcome issues of dependencies between inputs, before the data being passed through to an inout variable selection (IVS) stage. While we demonstrated previously with artificial data that ICA can prevent an overestimation of necessary input variables, we show here that mixing between input variables is common in real-world data sets so that ICA preprocessing is useful in practice. This experimental test is based on new measures introduced in this paper. Furthermore, we extend the implementation of our variable selection scheme to a statistical dependency test based on mutual information and test several algorithms on Gaussian and sub-Gaussian signals. Specifically, we propose a novel method of quantifying linear dependencies using ICA estimates of mixing matrices with a new Linear Mixing Measure (LMM).