Implementing plastic weights in neural networks using low precision arithmetic

Authors:
Christopher Johansson;Anders Lansner
Affiliations:
School of Computer Science and Communication, Royal Institute of Technology, Lindstedtsv. 3 Stockholm, 100 44, Sweden;School of Computer Science and Communication, Royal Institute of Technology, Lindstedtsv. 3 Stockholm, 100 44, Sweden
Venue:
Neurocomputing
Year:
2009

Citing 10
Cited 0

Bayesian neural networks with confidence estimations applied to data mining

Computational Statistics & Data Analysis
Machine Learning

Machine Learning
Introduction to Reinforcement Learning

Introduction to Reinforcement Learning
On the capabilities of neural networks using limited precision weights

Neural Networks
Efficient Event-Driven Simulation of Large Networks of Spiking Neurons and Dynamical Synapses

Neural Computation
Towards cortex sized artificial neural systems

Neural Networks
Imposing Biological Constraints onto an Abstract Neocortical Attractor Network Model

Neural Computation
Synaptic plasticity in spiking neural networks (SP2INN): a system approach

IEEE Transactions on Neural Networks
The TInMANN VLSI chip

IEEE Transactions on Neural Networks
Learning with limited numerical precision using the cascade-correlation algorithm

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.01

Visualization

Abstract

In this letter, we develop a fixed-point arithmetic, low precision, implementation of an exponentially weighted moving average (EWMA) that is used in a neural network with plastic weights. We analyze the proposed design both analytically and experimentally, and we also evaluate its performance in the application of an attractor neural network. The EWMA in the proposed design has a constant relative truncation error, which is important for avoiding round-off errors in applications with slowly decaying processes, e.g. connectionist networks. We conclude that the proposed design offers greatly improved memory and computational efficiency compared to a naive implementation of the EWMA's difference equation, and that it is well suited for implementation in digital hardware.