Feature selection with missing data using mutual information estimators

Authors:
Gauthier Doquire;Michel Verleysen
Affiliations:
Machine Learning Group-ICTEAM, Université catholique de Louvain, Place du Levant 3, 1348 Louvain-la-Neuve, Belgium;Machine Learning Group-ICTEAM, Université catholique de Louvain, Place du Levant 3, 1348 Louvain-la-Neuve, Belgium
Venue:
Neurocomputing
Year:
2012

Citing 17
Cited 3

Unknown attribute values in induction

Proceedings of the sixth international workshop on Machine learning
Wrappers for feature subset selection

Artificial Intelligence - Special issue on relevance
A Comparative Study on Feature Selection in Text Categorization

ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
An introduction to variable and feature selection

The Journal of Machine Learning Research
On the Kernel Widths in Radial-Basis Function Networks

Neural Processing Letters
Fast Binary Feature Selection with Conditional Mutual Information

The Journal of Machine Learning Research
Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy

IEEE Transactions on Pattern Analysis and Machine Intelligence
Incomplete-data classification using logistic regression

ICML '05 Proceedings of the 22nd international conference on Machine learning
Handling missing values in support vector machine classifiers

Neural Networks - 2005 Special issue: IJCNN 2005
Resampling methods for parameter-free and robust feature selection with mutual information

Neurocomputing
A review of feature selection techniques in bioinformatics

Bioinformatics
Combination of KNN-Based Feature Selection and KNNBased Missing-Value Imputation of Microarray Data

ICICIC '08 Proceedings of the 2008 3rd International Conference on Innovative Computing Information and Control
Information-theoretic feature selection for functional data classification

Neurocomputing
Representation of functional data in neural networks

Neurocomputing
Robust feature selection by mutual information distributions

UAI'02 Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence
Fuzzy c-means clustering of incomplete data

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Using mutual information for selecting features in supervised neural net learning

IEEE Transactions on Neural Networks

Feature selection techniques with class separability for multivariate time series

Neurocomputing
Distance estimation in numerical data sets with missing values

Information Sciences: an International Journal
Mixture of Gaussians for distance estimation with missing data

Neurocomputing

Quantified Score

Hi-index	0.01

Visualization

Abstract

Feature selection is an important preprocessing task for many machine learning and pattern recognition applications, including regression and classification. Missing data are encountered in many real-world problems and have to be considered in practice. This paper addresses the problem of feature selection in prediction problems where some occurrences of features are missing. To this end, the well-known mutual information criterion is used. More precisely, it is shown how a recently introduced nearest neighbors based mutual information estimator can be extended to handle missing data. This estimator has the advantage over traditional ones that it does not directly estimate any probability density function. Consequently, the mutual information may be reliably estimated even when the dimension of the space increases. Results on artificial as well as real-world datasets indicate that the method is able to select important features without the need for any imputation algorithm, under the assumption of missing completely at random data. Moreover, experiments show that selecting the features before imputing the data generally increases the precision of the prediction models, in particular when the proportion of missing data is high.