CAVIAR: a 45k neuron, 5M synapse, 12G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking

Authors:
Rafael Serrano-Gotarredona;Matthias Oster;Patrick Lichtsteiner;Alejandro Linares-Barranco;Rafael Paz-Vicente;Francisco Gómez-Rodríguez;Luis Camuñas-Mesa;Raphael Berner;Manuel Rivas-Pérez;Tobi Delbrück;Shih-Chii Liu;Rodney Douglas;Philipp Häfliger;Gabriel Jiménez-Moreno;Anton Civit Ballcels;Teresa Serrano-Gotarredona;Antonio J. Acosta-Jiménez;Bernabé Linares-Barranco
Affiliations:
Austriamicrosystems, Valencia, Spain and Consejo Superior de Investigaciones Cientificas, Seville Microelectronics Institute, Seville, Spain;Varian Medical Systems, Baden, Switzerland and Institute of Neuroinformatics, ETH Zürich, University of Zürich, Zürich, Switzerland;Espros Photonics Corporation, Baar, Switzerland and Institute of Neuroinformatics, ETH Zürich, University of Zürich, Zürich, Switzerland;Computer Architecture and Technology Department, University of Seville, Seville, Spain;Computer Architecture and Technology Department, University of Seville, Seville, Spain;Computer Architecture and Technology Department, University of Seville, Seville, Spain;Consejo Superior de Investigaciones Cientificas, Seville Microelectronics Institute, Seville, Spain;Institute of Neuroinformatics, ETH Zürich, University of Zürich, Zürich, Switzerland;Computer Architecture and Technology Department, University of Seville, Seville, Spain;Institute of Neuroinformatics, ETH Zürich, University of Zürich, Zürich, Switzerland;Institute of Neuroinformatics, ETH Zürich, University of Zürich, Zürich, Switzerland;Institute of Neuroinformatics, ETH Zürich, University of Zürich, Zürich, Switzerland;Informatics, University of Oslo, Oslo;Computer Architecture and Technology Department, University of Seville, Seville, Spain;Computer Architecture and Technology Department, University of Seville, Seville, Spain;Consejo Superior de Investigaciones Cientificas, Seville Microelectronics Institute, Seville, Spain;Consejo Superior de Investigaciones Cientificas, Seville Microelectronics Institute, Seville, Spain;Consejo Superior de Investigaciones Cientificas, Seville Microelectronics Institute, Seville, Spain
Venue:
IEEE Transactions on Neural Networks
Year:
2009

Citing 28
Cited 16

Self-organization and associative memory: 3rd edition

Self-organization and associative memory: 3rd edition
Synthetic neural systems in VLSI

An introduction to neural and electronic networks
Wiring considerations in analog VLSI systems, with application to field-programmable networks

Wiring considerations in analog VLSI systems, with application to field-programmable networks
Scanners for visualizing activity of analog VLSI circuitry

Analog Integrated Circuits and Signal Processing
VLSI analogs of neuronal visual processing: a synthesis of form and function

VLSI analogs of neuronal visual processing: a synthesis of form and function
Synthetic aperture radar processing by a multiple scale neural system for boundary and surface representation

Neural Networks - Special issue: automatic target recognition
Dynamical cell assembly hypothesis—theoretical possibility of spatio-temporal coding in the cortex

Neural Networks - 1996 Special issue: four major hypotheses in neuroscience
Communicating neuronal ensembles between neuromorphic chips

Neuromorphic systems engineering
Convolutional networks for images, speech, and time series

The handbook of brain theory and neural networks
A neural network for enhancing boundaries and surfaces in synthetic aperture radar images

Neural Networks
Finding circles by an array of accumulators

Communications of the ACM
An Analog VLSI System for Stereoscopic Vision

An Analog VLSI System for Stereoscopic Vision
Asynchronous Communication of 2D Motion Information UsingWinner-Takes-All Arbitration

Analog Integrated Circuits and Signal Processing
Modeling short-term synaptic depression in Silicon

Neural Computation
The time-rescaling theorem and its application to neural spike train data analysis

Neural Computation
A multi-sender asynchronous extension to the AER protocol

ARVLSI '95 Proceedings of the 16th Conference on Advanced Research in VLSI (ARVLSI'95)
Motion Vision Sensor Architecture with Asynchronous Self-Signaling Pixels

CAMP '97 Proceedings of the 1997 Computer Architectures for Machine Perception (CAMP '97)
A Reconfigurable Neuromorphic VLSI Multi-Chip System Applied to Visual Motion Computation

MICRONEURO '99 Proceedings of the 7th International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems
Retinomorphic Chips that see Quadrupple Images

MICRONEURO '99 Proceedings of the 7th International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems
Inter-spike-intervals analysis of AER Poisson-like generator hardware

Neurocomputing
Computation with spikes in a winner-take-all network

Neural Computation
Computing with inter-spike interval codes in networks of integrate and fire neurons

Neurocomputing
Two hardware implementations of the exhaustive synthetic AER generation method

IWANN'05 Proceedings of the 8th international conference on Artificial Neural Networks: computational Intelligence and Bioinspired Systems
Face recognition: a convolutional neural-network approach

IEEE Transactions on Neural Networks
Evaluation of convolutional neural networks for visual recognition

IEEE Transactions on Neural Networks
On algorithmic rate-coded AER generation

IEEE Transactions on Neural Networks
Adaptive WTA With an Analog VLSI Neuromorphic Learning Chip

IEEE Transactions on Neural Networks
On Real-Time AER 2-D Convolutions Hardware for Neuromorphic Spike-Based Cortical Processing

IEEE Transactions on Neural Networks

Fast vision through frameless event-based sensing and convolutional processing: application to texture recognition

IEEE Transactions on Neural Networks
Multilayer processing of spatiotemporal spike patterns in a neuron with active dendrites

Neural Computation
Human tracking using convolutional neural networks

IEEE Transactions on Neural Networks
An AER to CAN bridge for spike-based robot control

IWANN'11 Proceedings of the 11th international conference on Artificial neural networks conference on Advances in computational intelligence - Volume Part I
Frequency analysis of a 64×64 pixel retinomorphic system with AER output to estimate the limits to apply onto specific mechanical environment

IWANN'11 Proceedings of the 11th international conference on Artificial neural networks conference on Advances in computational intelligence - Volume Part I
An AER spike-processing filter simulator and automatic VHDL generator based on cellular automata

IWANN'11 Proceedings of the 11th international conference on Artificial neural networks conference on Advances in computational intelligence - Volume Part I
Simulating building blocks for spikes signals processing

IWANN'11 Proceedings of the 11th international conference on Artificial neural networks conference on Advances in computational intelligence - Volume Part II
On the designing of spikes band-pass filters for FPGA

ICANN'11 Proceedings of the 21st international conference on Artificial neural networks - Volume Part II
A 32GBit/s communication SoC for a waferscale neuromorphic system

Integration, the VLSI Journal
A systematic method for configuring vlsi networks of spiking neurons

Neural Computation
Asynchronous frameless event-based optical flow

Neural Networks
Silicon spiking neurons for hardware implementation of extreme learning machines

Neurocomputing
Fun with asynchronous vision sensors and processing

ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume Part I
A real-time, event-driven neuromorphic system for goal-directed attentional selection

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part II
Continuous real-world inputs can open up alternative accelerator designs

Proceedings of the 40th Annual International Symposium on Computer Architecture
2013 Special Issue: Design of silicon brains in the nano-CMOS era: Spiking neurons, learning synapses and neural architecture optimization

Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper describes CAVIAR, a massively parallel hardware implementation of a spike-based sensing-processing-learning-actuating system inspired by the physiology of the nervous system. CAVIAR uses the asychronous address-event representation (AER) communication framework and was developed in the context of a European Union funded project. It has four custom mixed-signal AER chips, five custom digital AER interface components, 45k neurons (spiking cells), up to 5M synapses, performs 12G synaptic operations per second, and achieves millisecond object recognition and tracking latencies.