A general approach for mutual information minimization and its application to blind source separation

Authors:
Massoud Babaie-Zadeh;Christian Jutten
Affiliations:
Electrical Engineering Department, Sharif University of Technology, Tehran, Iran;Laboratoire des Images et des Signaux (LIS), Institut National Polytechnique de Grenoble and Université Joseph Fourier, Grenoble, France
Venue:
Signal Processing - Special issue: Information theoretic signal processing
Year:
2005

Citing 14
Cited 8

Introduction to statistical pattern recognition (2nd ed.)

Introduction to statistical pattern recognition (2nd ed.)
Elements of information theory

Elements of information theory
Independent component analysis, a new concept?

Signal Processing - Special issue on higher order statistics
Natural gradient works efficiently in learning

Neural Computation
An unsupervised ensemble learning method for nonlinear dynamic state-space models

Neural Computation
Entropy Optimization - Application to Blind Source Separation

ICANN '97 Proceedings of the 7th International Conference on Artificial Neural Networks
Separating Convolutive Mixtures by Mutual Information Minimization

IWANN '01 Proceedings of the 6th International Work-Conference on Artificial and Natural Neural Networks: Bio-inspired Applications of Connectionism-Part II
ICA using spacings estimates of entropy

The Journal of Machine Learning Research
Three easy ways for separating nonlinear mixtures?

Signal Processing - Special issue on independent components analysis and beyond
MISEP - Linear and nonlinear ICA based on mutual information

The Journal of Machine Learning Research
On the entropy minimization of a linear mixture of variables for source separation

Signal Processing - Special issue: Information theoretic signal processing
Source separation in post-nonlinear mixtures

IEEE Transactions on Signal Processing
Equivariant adaptive source separation

IEEE Transactions on Signal Processing
Mutual information approach to blind separation of stationary sources

IEEE Transactions on Information Theory

On the entropy minimization of a linear mixture of variables for source separation

Signal Processing - Special issue: Information theoretic signal processing
Quasi-optimal EASI algorithm based on the Score Function Difference (SFD)

Neurocomputing
Letters: Adaptive filtering under maximum mutual information criterion

Neurocomputing
Noise speech wavelet analyzing in special time ranges

ICACT'10 Proceedings of the 12th international conference on Advanced communication technology
Noise wavelet evaluating near to zero by thresholding method analyzing

ICCC'11 Proceedings of the 2011 international conference on Computers and computing
Separability of convolutive mixtures: application to the separation of combustion noise and piston-slap in diesel engine

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Recurrent networks for separating extractable-target nonlinear mixtures. Part II. Blind configurations

Signal Processing
A new histogram-based estimation technique of entropy and mutual information using mean squared error minimization

Computers and Electrical Engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, a nonparametric "gradient" of the mutual information is first introduced. It is used for showing that mutual information has no local minima. Using the introduced "gradient", two general gradient based approaches for minimizing mutual information in a parametric model are then presented. These approaches are quite general, and principally they can be used in any mutual information minimization problem. In blind source separation, these approaches provide powerful tools for separating any complicated (yet separable) mixing model. In this paper, they are used to develop algorithms for separating four separable mixing models: linear instantaneous, linear convolutive, post nonlinear (PNL) and convolutive post nonlinear (CPNL) mixtures.