Information processing in dendrites-II.: information theoretic complexity

Authors:
Kevin N. Gurney
Affiliations:
Department of Psychology, University of Sheffield, Sheffield, S10 2TP UK
Venue:
Neural Networks
Year:
2001

Citing 6
Cited 9

Self-Organization in a Perceptual Network

Computer
Elements of information theory

Elements of information theory
Original Contribution: Efficient information transfer and anti-Hebbian neural networks

Neural Networks
Spikes: exploring the neural code

Spikes: exploring the neural code
Information-Theoretic Limitations of Formal Systems

Journal of the ACM (JACM)
Observer-participant models of neural processing

IEEE Transactions on Neural Networks

Predicting the behaviour of G-RAM networks

Neural Networks
Third-order generalization: A new approach to categorizing higher-order generalization

Neurocomputing
An information-theoretic approach to feature extraction in competitive learning

Neurocomputing
Non-stationary and stationary prediction of financial time series using dynamic ridge polynomial neural network

Neurocomputing
Structural enhanced information to detect features in competitive learning

SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Structural enhanced information and its application to improved visualization of self-organizing maps

Applied Intelligence
Selective information enhancement learning for creating interpretable representations in competitive learning

Neural Networks
Relative information maximization and its application to the extraction of explicit class structure in SOM

Neurocomputing
Double enhancement learning for explicit internal representations: unifying self-enhancement and information enhancement to incorporate information on input variables

Applied Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

In the companion paper, we established a rationale for exploring the general principles of dendritic processing using a class of Boolean functions--the Multi-Cube Units (MCUs). Here, we use this approach to further characterise dendritic processing using ideas from information theory and studies in complexity. The starting point is a novel decomposition of a Boolean function's total mutual information (between input variables and the output). Each component of the decomposition is a mutual information measure with respect to a single input, conditioned on a subset of the remaining inputs. We call this decomposition the information spectrum and conceive of it as a re-representation of the function in the information domain. Furthermore, the information spectrum of a Boolean function may be assigned a complexity value using the approximate entropy introduced by Pincus (Pincus, S. M. (1991). Approximate entropy as a measure of system complexity. Proc. Natl. Acad. Sci. USA, 88, 2297-2301). Using Monte Carlo methods, we provide evidence that the information spectral complexity of MCUs is larger than that of any other class of Boolean function. we explain this phenomenon in terms of information flow through the 2-stage MCU architecture. Under our modelling assumptions, the implication for biological neural processing is that dendrites implement functions that have maximal information spectral complexity with respect to the class of multivariate functions from which they are drawn.