Feature selection, mutual information, and the classification of high-dimensional patterns: Applications to image classification and microarray data analysis

Authors:
Boyan Bonev;Francisco Escolano;Miguel Cazorla
Affiliations:
Universidad de Alicante, Deptartamento Ciencia Computación e Inteligencia Artificial, Ap. Correos 99, 03080, Alicante, Spain;Universidad de Alicante, Deptartamento Ciencia Computación e Inteligencia Artificial, Ap. Correos 99, 03080, Alicante, Spain;Universidad de Alicante, Deptartamento Ciencia Computación e Inteligencia Artificial, Ap. Correos 99, 03080, Alicante, Spain
Venue:
Pattern Analysis & Applications - Special Issue: Non-parametric distance-based classification techniques and their applications
Year:
2008

Citing 0
Cited 8

Normalized mutual information feature selection

IEEE Transactions on Neural Networks
Ensemble gene selection for cancer classification

Pattern Recognition
On Guo and Nixon's criterion for feature subset selection: assumptions, implications, and alternative options

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans - Special issue on recent advances in biometrics
High-dimensional spectral feature selection for 3D object recognition based on reeb graphs

SSPR&SPR'10 Proceedings of the 2010 joint IAPR international conference on Structural, syntactic, and statistical pattern recognition
Feature selection using hierarchical feature clustering

Proceedings of the 20th ACM international conference on Information and knowledge management
UPM-3D facial expression recognition Database(UPM-3DFE)

PRICAI'12 Proceedings of the 12th Pacific Rim international conference on Trends in Artificial Intelligence
Information-theoretic selection of high-dimensional spectral features for structural recognition

Computer Vision and Image Understanding
Optimized dissimilarity space embedding for labeled graphs

Information Sciences: an International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose a novel feature selection filter for supervised learning, which relies on the efficient estimation of the mutual information between a high-dimensional set of features and the classes. We bypass the estimation of the probability density function with the aid of the entropic-graphs approximation of Rényi entropy, and the subsequent approximation of the Shannon entropy. Thus, the complexity does not depend on the number of dimensions but on the number of patterns/samples, and the curse of dimensionality is circumvented. We show that it is then possible to outperform algorithms which individually rank features, as well as a greedy algorithm based on the maximal relevance and minimal redundancy criterion. We successfully test our method both in the contexts of image classification and microarray data classification. For most of the tested data sets, we obtain better classification results than those reported in the literature.