Feedback in multimodal self-organizing networks enhances perception of corrupted stimuli

Authors:
Andrew P. Papliński;Lennart Gustafsson
Affiliations:
Clayton School of Information Technology, Monash University, Victoria, Australia;Computer Science and Electrical Engineering, Luleå University of Technology, Luleå, Sweden
Venue:
AI'06 Proceedings of the 19th Australian joint conference on Artificial Intelligence: advances in Artificial Intelligence
Year:
2006

Citing 7
Cited 4

The 'Neural' Phonetic Typewriter

Computer
Speech and Audio Signal Processing: Processing and Perception of Speech and Music

Speech and Audio Signal Processing: Processing and Perception of Speech and Music
Self-Organizing Maps

Self-Organizing Maps
Reading Speech from Still and Moving Faces: The Neural Substrates of Visible Speech

Journal of Cognitive Neuroscience
Neural Specialization for Letter Recognition

Journal of Cognitive Neuroscience
The Fusiform "Face Area" is Part of a Network that Processes Faces at the Individual Level

Journal of Cognitive Neuroscience
Multimodal feedforward self-organizing maps

CIS'05 Proceedings of the 2005 international conference on Computational Intelligence and Security - Volume Part I

A multimodal self-organizing network for sensory integration of letters and phonemes

ASC '07 Proceedings of The Eleventh IASTED International Conference on Artificial Intelligence and Soft Computing
Rotation invariant categorization of visual objects using radon transform and self-organizing modules

ICONIP'10 Proceedings of the 17th international conference on Neural information processing: models and applications - Volume Part II
A self-organized artificial neural network architecture for sensory integration with applications to letter-phoneme integration

Neural Computation
A recurrent multimodal network for binding written words and sensory-based semantics into concepts

ICONIP'11 Proceedings of the 18th international conference on Neural Information Processing - Volume Part I

Quantified Score

Hi-index	0.00

Visualization

Abstract

It is known from psychology and neuroscience that multimodal integration of sensory information enhances the perception of stimuli that are corrupted in one or more modalities. A prominent example of this is that auditory perception of speech is enhanced when speech is bimodal, i.e. when it also has a visual modality. The function of the cortical network processing speech in auditory and visual cortices and in multimodal association areas, is modeled with a Multimodal Self-Organizing Network (MuSON), consisting of several Kohonen Self-Organizing Maps (SOM) with both feedforward and feedback connections. Simulations with heavily corrupted phonemes and uncorrupted letters as inputs to the MuSON demonstrate a strongly enhanced auditory perception. This is explained by feedback from the bimodal area into the auditory stream, as in cortical processing.