Low bias histogram-based estimation of mutual information for feature selection

Authors:
Abdenour Hacine-Gharbi;Philippe Ravier;Rachid Harba;Tayeb Mohamadi
Affiliations:
PRISME laboratory, BP 6744 Orléans Cedex 2, University of Orléans, France and LMSE Laboratory, Electronics Department, The University Center of Bordj Bou Arréridj, Elanasser-Bordj B ...;PRISME laboratory, BP 6744 Orléans Cedex 2, University of Orléans, France;PRISME laboratory, BP 6744 Orléans Cedex 2, University of Orléans, France;Electronics Department, Faculty of technology, Ferhat Abbas University, Setif 19000, Algeria
Venue:
Pattern Recognition Letters
Year:
2012

Citing 11
Cited 1

Statistical Pattern Recognition: A Review

IEEE Transactions on Pattern Analysis and Machine Intelligence
Relevance of time-frequency features for phonetic and speaker-channel classification

Speech Communication
Input Feature Selection by Mutual Information Based on Parzen Window

IEEE Transactions on Pattern Analysis and Machine Intelligence
Resampling methods for parameter-free and robust feature selection with mutual information

Neurocomputing
A Method for Selecting the Bin Size of a Time Histogram

Neural Computation
Relevance measures for subset variable selection in regression problems based on k-additive mutual information

Computational Statistics & Data Analysis
Information theoretic feature extraction for audio-visual speech recognition

IEEE Transactions on Signal Processing
Textural feature selection by joint mutual information based on Gaussian mixture model for multispectral image classification

Pattern Recognition Letters
On estimating mutual information for feature selection

ICANN'10 Proceedings of the 20th international conference on Artificial neural networks: Part I
Static and Dynamic Spectral Features: Their Noise Robustness and Optimal Weights for ASR

IEEE Transactions on Audio, Speech, and Language Processing
Estimation of the information by an adaptive partitioning of the observation space

IEEE Transactions on Information Theory

A new histogram-based estimation technique of entropy and mutual information using mean squared error minimization

Computers and Electrical Engineering

Quantified Score

Hi-index	0.10

Visualization

Abstract

This paper presents a low bias histogram-based estimation of mutual information and its application to feature selection problems. By canceling the first order bias, the estimation avoids the bias accumulation problem that affects classical methods. As a consequence, on a synthetic feature selection problem, only the proposed method results in the exact number of features to be chosen in the Gaussian case when compared to four other approaches. In a speech recognition application, the proposed method and the Sturges method are the only ones that lead to a correct number of selected features in the noise free case. In the reduced data case, only the proposed method points out the optimal number of features to select. Finally, in the noisy case, only the proposed method leads to results of high quality; other methods show severely underestimated numbers of selected features.