Optimizing neural networks on SIMD parallel computers

Authors:
Andrea Di Blas;Arun Jagota;Richard Hughey
Affiliations:
Department of Computer Engineering, Baskin School of Engineering, University of California, Santa Cruz, United States;Department of Computer Engineering, Baskin School of Engineering, University of California, Santa Cruz, United States;Department of Computer Engineering, Baskin School of Engineering, University of California, Santa Cruz, United States
Venue:
Parallel Computing
Year:
2005

Citing 12
Cited 7

Algorithmic mapping of neural network Models onto Parallel SIMD Machines

IEEE Transactions on Computers - Special issue on artificial neural networks
A sublinear-time randomized parallel algorithm for the maximum clique problem in perfect graphs

SODA '91 Proceedings of the second annual ACM-SIAM symposium on Discrete algorithms
Exploiting Domain-Specific Properties: Compiling Parallel Dynamic Neural Network Algorithms into Efficient Code

IEEE Transactions on Parallel and Distributed Systems
On the Efficiency of Parallel Backtracking

IEEE Transactions on Parallel and Distributed Systems
Asynchronous Problems on SIMD Parallel Computers

IEEE Transactions on Parallel and Distributed Systems
Neural Networks for Combinatorial Optimization: a Review of More Than a Decade of Research

INFORMS Journal on Computing
Explicit SIMD Programming for Asynchronous Applications

ASAP '00 Proceedings of the IEEE International Conference on Application-Specific Systems, Architectures, and Processors
Kestrel: Design of an 8-bit SIMD Parallel Processor

ARVLSI '97 Proceedings of the 17th Conference on Advanced Research in VLSI (ARVLSI '97)
MIMD programs on SIMD architectures

FRONTIERS '96 Proceedings of the 6th Symposium on the Frontiers of Massively Parallel Computation
Solving the Maximum Clique Problem using PUBB

IPPS '98 Proceedings of the 12th. International Parallel Processing Symposium on International Parallel Processing Symposium
Energy function-based approaches to graph coloring

IEEE Transactions on Neural Networks
Approximating maximum clique with a Hopfield network

IEEE Transactions on Neural Networks

The UCSC Kestrel Parallel Processor

IEEE Transactions on Parallel and Distributed Systems
Discrete Hopfield network combined with estimation of distribution for unconstrained binary quadratic programming problem

Expert Systems with Applications: An International Journal
Simulating biological-inspired spiking neural networks with OpenCL

ICANN'10 Proceedings of the 20th international conference on Artificial neural networks: Part I
Improved stochastic competitive Hopfield network for polygonal approximation

Expert Systems with Applications: An International Journal
An approximate method for filtering out data dependencies with a sufficiently large distance between memory references

The Journal of Supercomputing
A parallel evolving algorithm for flexible neural tree

Parallel Computing
Combining tabu Hopfield network and estimation of distribution for unconstrained binary quadratic programming problem

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

Hopfield neural networks are often used to solve difficult combinatorial optimization problems. Multiple restarts versions find better solutions but are slow on serial computers. Here, we study two parallel implementations on SIMD computers of multiple restarts Hopfield networks for solving the maximum clique problem. The first one is a fine-grained implementation on the Kestrel Parallel Processor, a linear SIMD array designed and built the University of California, Santa Cruz. The second one is an implementation on the MasPar MP-2 according to the ''SIMD Phase Programming Model'', a new method to solve asynchronous, irregular problems on SIMD machines. We find that the neural networks map well to the parallel architectures and afford substantial speedups with respect to the serial program, without sacrificing solution quality.