Elements of information theory
Elements of information theory
Statistical Pattern Recognition: A Review
IEEE Transactions on Pattern Analysis and Machine Intelligence
Input Feature Selection by Mutual Information Based on Parzen Window
IEEE Transactions on Pattern Analysis and Machine Intelligence
Signal Processing - Special issue: Information theoretic signal processing
Computational Statistics & Data Analysis
Information theoretic feature extraction for audio-visual speech recognition
IEEE Transactions on Signal Processing
Estimating redundancy information of selected features in multi-dimensional pattern classification
Pattern Recognition Letters
Mutual Information Analysis: a Comprehensive Study
Journal of Cryptology - Special Issue on Hardware and Security
Low bias histogram-based estimation of mutual information for feature selection
Pattern Recognition Letters
Estimation of the information by an adaptive partitioning of the observation space
IEEE Transactions on Information Theory
Advanced search algorithms for information-theoretic learning with kernel-based estimators
IEEE Transactions on Neural Networks
Hi-index | 0.00 |
Mutual Information (MI) has extensively been used as a measure of similarity or dependence between random variables (or parameters) in different signal and image processing applications. However, MI estimation techniques are known to exhibit a large bias, a high Mean Squared Error (MSE), and can computationally be very costly. In order to overcome these drawbacks, we propose here a novel fast and low MSE histogram-based estimation technique for the computation of entropy and the mutual information. By minimizing the MSE, the estimation avoids the error accumulation problem of traditional methods. We derive an expression for the optimal number of bins to estimate the MI for both continuous and discrete random variables. Experimental results from a speech recognition problem and a computer aided diagnosis problem show the power of the proposed approach in estimating the optimal number of selected features with enhanced classification results compared to existing approaches.