An efficient hardware architecture for a neural network activation function generator

Authors:
Daniel Larkin;Andrew Kinane;Valentin Muresan;Noel O’Connor
Affiliations:
Centre for Digital Video Processing, Dublin City University, Dublin 9, Ireland;Centre for Digital Video Processing, Dublin City University, Dublin 9, Ireland;Centre for Digital Video Processing, Dublin City University, Dublin 9, Ireland;Centre for Digital Video Processing, Dublin City University, Dublin 9, Ireland
Venue:
ISNN'06 Proceedings of the Third international conference on Advances in Neural Networks - Volume Part III
Year:
2006

Citing 9
Cited 4

Neural networks: applications in industry, business and science

Communications of the ACM
Computer Arithmetic Algorithms

Computer Arithmetic Algorithms
Neural Networks: A Comprehensive Foundation

Neural Networks: A Comprehensive Foundation
Learning on Silicon: Adaptive VLSI Neural Systems

Learning on Silicon: Adaptive VLSI Neural Systems
On the capabilities of neural networks using limited precision weights

Neural Networks
ULSI Architectures for Artificial Neural Networks

IEEE Micro
Finite Precision Error Analysis of Neural Network Hardware Implementations

IEEE Transactions on Computers
High-Speed Function Approximation Using a Minimax Quadratic Interpolator

IEEE Transactions on Computers
Elementary function generators for neural-network emulators

IEEE Transactions on Neural Networks

Parallel tracing of multiple trajectories in gradient descent algorithm with Cell Broadband Engine

SpringSim '09 Proceedings of the 2009 Spring Simulation Multiconference
Towards hardware acceleration of neuroevolution for multimedia processing applications on mobile devices

ICONIP'06 Proceedings of the 13th international conference on Neural information processing - Volume Part III
A Fully Pipelined FPGA Architecture of a Factored Restricted Boltzmann Machine Artificial Neural Network

ACM Transactions on Reconfigurable Technology and Systems (TRETS)
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning

Proceedings of the 19th international conference on Architectural support for programming languages and operating systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper proposes an efficient hardware architecture for a function generator suitable for an artificial neural network (ANN). A spline-based approximation function is designed that provides a good trade-off between accuracy and silicon area, whilst also being inherently scalable and adaptable for numerous activation functions. This has been achieved by using a minimax polynomial and through optimal placement of the approximating polynomials based on the results of a genetic algorithm. The approximation error of the proposed method compares favourably to all related research in this field. Efficient hardware multiplication circuitry is used in the implementation, which reduces the area overhead and increases the throughput.