An information-maximization approach to blind separation and blind deconvolution

  • Authors:
  • Anthony J. Bell;Terrence J. Sejnowski

  • Affiliations:
  • -;-

  • Venue:
  • Neural Computation
  • Year:
  • 1995

Quantified Score

Hi-index 0.03

Visualization

Abstract

We derive a new self-organizing learning algorithm thatmaximizes the information transferred in a network of nonlinearunits. The algorithm does not assume any knowledge of the inputdistributions, and is defined here for the zero-noise limit. Underthese conditions, information maximization has extra properties notfound in the linear case (Linsker 1989). The nonlinearities in thetransfer function are able to pick up higher-order moments of theinput distributions and perform something akin to true redundancyreduction between units in the output representation. This enablesthe network to separate statistically independent components in theinputs: a higher-order generalization of principal componentsanalysis. We apply the network to the source separation (orcocktail party) problem, successfully separating unknown mixturesof up to 10 speakers. We also show that a variant on the networkarchitecture is able to perform blind deconvolution (cancellationof unknown echoes and reverberation in a speech signal). Finally,we derive dependencies of information transfer on time delays. Wesuggest that information maximization provides a unifying frameworkfor problems in "blind" signal processing.