Original Contribution: Efficient information transfer and anti-Hebbian neural networks

Authors:
Mark D. Plumbley
Affiliations:
-
Venue:
Neural Networks
Year:
1993

Citing 5
Cited 5

Self-Organization in a Perceptual Network

Computer
Adaptation and decorrelation in the cortex

The computing neuron
The representation and storage of information in neural networks in the primate cerebral cortex and hippocampus

The computing neuron
Convergent algorithm for sensory receptive field development

Neural Computation
Towards a theory of early visual processing

Neural Computation

Information processing in dendrites-II.: information theoretic complexity

Neural Networks
Derivation of the visual contrast response function by maximizing information rate

Neural Computation
Nonlinear Complex-Valued Extensions of Hebbian Learning: An Essay

Neural Computation
Factor Analysis Using Delta-Rule Wake-Sleep Learning

Neural Computation
The infomin criterion: an information theoretic unifying objective function for topographic mappings

ICANN/ICONIP'03 Proceedings of the 2003 joint international conference on Artificial neural networks and neural information processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

We attempt to maximise mutual information in a neural network where the output signal is corrupted by noise in the transmission process to the next stage. We assume that the average power available for transmission is limited and that the transmission noise is uncorrelated equal-variance Gaussian noise. Using the Lagrange multiplier technique, we construct a utility function that should be maximised to achieve our optimum. This optimum will be achieved when the output signal components are also uncorrelated and of equal variance. For a linear network with lateral inhibitory connections, using an anti-Hebbian algorithm with modified self-inhibitory connections we guarantee to increase our utility function over time and converge to the required optimum, although not by steepest ascent. For a network with inhibitory interneurons, a combined Hebbian/anti-Hebbian algorithm will similarly increase the utility function over time to find the optimum. This network with interneurons suggests that negative feedback may be used in a perceptual system to optimise forward transmission of information.