Audio-visual localization with hierarchical topographic maps: Modeling the superior colliculus

Authors:
M. C. Casey;A. Pavlou;A. Timotheou
Affiliations:
Department of Computing, University of Surrey, Guildford, Surrey GU2 7XH, UK;Department of Computing, University of Surrey, Guildford, Surrey GU2 7XH, UK;Department of Computing, University of Surrey, Guildford, Surrey GU2 7XH, UK
Venue:
Neurocomputing
Year:
2012

Citing 18
Cited 0

Self-Organization in a Perceptual Network

Computer
Feature discovery by competitive learning

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
A Model of Saliency-Based Visual Attention for Rapid Scene Analysis

IEEE Transactions on Pattern Analysis and Machine Intelligence
Modeling cross-modal enhancement and modality-specific suppression in multisensory neurons

Neural Computation
Using Bayes’ Rule to Model Multisensory Enhancement in the Superior Colliculus

Neural Computation
Distinguishing Causal Interactions in Neural Populations

Neural Computation
A neural model of decision-making by the superior colicullus in an antisaccade task

Neural Networks
Behavioral indices of multisensory integration: Orientation to visual cues is affected by auditory stimuli

Journal of Cognitive Neuroscience
2008 Special Issue: A theoretical study of multisensory integration in the superior colliculus by a neural network model

Neural Networks
A head-neck-eye system that learns fault-tolerant saccades to 3-D targets using a self-organizing neural model

Neural Networks
The adaptation of visual and auditory integration in the barn owl superior colliculus with Spike Timing Dependent Plasticity

Neural Networks
2009 Special Issue: Cortext: A columnar model of bottom-up and top-down processing in the neocortex

Neural Networks
Identifying emotions using topographic conditioning maps

ICONIP'08 Proceedings of the 15th international conference on Advances in neuro-information processing - Volume Part I
A world survey of artificial brain projects, Part I: Large-scale brain simulations

Neurocomputing
A world survey of artificial brain projects, Part II: Biologically inspired cognitive architectures

Neurocomputing
A biologically inspired spiking neural network model of the auditory midbrain for sound source localisation

Neurocomputing
Beobot 2.0: Cluster architecture for mobile robotics

Journal of Field Robotics
Learning Bimodal Structure in Audio–Visual Data

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

A key attribute of the brain is its ability to seamlessly integrate sensory information to form a multisensory representation of the world. In early perceptual processing, the superior colliculus (SC) takes a leading role in integrating visual, auditory and somatosensory stimuli in order to direct eye movements. The SC forms a representation of multisensory space through a layering of retinotopic maps which are sensitive to different types of stimuli. These eye-centered topographic maps can adapt to crossmodal stimuli so that the SC can automatically shift our gaze, moderated by cortical feedback. In this paper we describe a neural network model of the SC consisting of a hierarchy of nine topographic maps that combine to form a multisensory retinotopic representation of audio-visual space. Our motivation is to evaluate whether a biologically plausible model of the SC can localize audio-visual inputs live from a camera and two microphones. We use spatial contrast and a novel form of temporal contrast for visual sensitivity, and interaural level difference for auditory sensitivity. Results are comparable with the performance observed in cats where coincident stimuli are accurately localized, while presentation of disparate stimuli causes a significant drop in performance. The benefit of crossmodal localization is shown by adding increasing amounts of noise to the visual stimuli to the point where audio-visual localization significantly out performs visual-only localization. This work demonstrates how a novel, biologically motivated model of low level multisensory processing can be applied to practical, real-world input in real-time, while maintaining its comparability with biology.