A novel information theory method for filter feature selection

Authors:
Boyan Bonev;Francisco Escolano;Miguel Angel Cazorla
Affiliations:
Department of Computer Science and Artificial Intelligence, Alicante University, Alicante, Spain;Department of Computer Science and Artificial Intelligence, Alicante University, Alicante, Spain;Department of Computer Science and Artificial Intelligence, Alicante University, Alicante, Spain
Venue:
MICAI'07 Proceedings of the artificial intelligence 6th Mexican international conference on Advances in artificial intelligence
Year:
2007

Citing 12
Cited 1

Elements of information theory

Elements of information theory
Selection of relevant features and examples in machine learning

Artificial Intelligence - Special issue on relevance
Independent component analysis: algorithms and applications

Neural Networks
Feature selection for high-dimensional genomic microarray data

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Estimation of entropy and mutual information

Neural Computation
Alignment by maximization of mutual information

ICCV '95 Proceedings of the Fifth International Conference on Computer Vision
Rényi Extrapolation of Shannon Entropy

Open Systems & Information Dynamics
An introduction to variable and feature selection

The Journal of Machine Learning Research
Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy

IEEE Transactions on Pattern Analysis and Machine Intelligence
EBEM: An Entropy-based EM Algorithm for Gaussian Mixture Models

ICPR '06 Proceedings of the 18th International Conference on Pattern Recognition - Volume 02
What should be expected from feature selection in small-sample settings

Bioinformatics
Asymptotic theory of greedy approximations to minimal k-point random graphs

IEEE Transactions on Information Theory

Feature Selection Using Artificial Neural Networks

MICAI '08 Proceedings of the 7th Mexican International Conference on Artificial Intelligence: Advances in Artificial Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we propose a novel filter for feature selection. Such filter relies on the estimation of the mutual information between features and classes. We bypass the estimation of the probability density function with the aid of the entropic-graphs approximation of Rényi entropy, and the subsequent approximation of the Shannon one. The complexity of such bypassing process does not depend on the number of dimensions but on the number of patterns/samples, and thus the curse of dimensionality is circumvented. We show that it is then possible to outperform a greedy algorithm based on the maximal relevance and minimal redundancy criterion. We successfully test our method both in the contexts of image classification and microarray data classification.