An efficient implementation of the back-propagation algorithm on the connection machine CM-2
Advances in neural information processing systems 2
Eli: a complete, flexible compiler construction system
Communications of the ACM
Factoring: a method for scheduling parallel loops
Communications of the ACM
Implementation of a portable nested data-parallel language
PPOPP '93 Proceedings of the fourth ACM SIGPLAN symposium on Principles and practice of parallel programming
Automatic array alignment in data-parallel programs
POPL '93 Proceedings of the 20th ACM SIGPLAN-SIGACT symposium on Principles of programming languages
Compiling machine-independent parallel programs
ACM SIGPLAN Notices
Automatic alignment of array data and processes to reduce communication time on DMPPs
PPOPP '95 Proceedings of the fifth ACM SIGPLAN symposium on Principles and practice of parallel programming
SYNAPSE-1: a high-speed general purpose parallel neurocomputer system
IPPS '95 Proceedings of the 9th International Symposium on Parallel Processing
A Parallel Programming Model for Irregular Dynamic Neural Networks
MPPM '97 Proceedings of the Conference on Massively Parallel Programming Models
Mapping connectionist networks onto parallel machines: a library approach
Mapping connectionist networks onto parallel machines: a library approach
Optimizing neural networks on SIMD parallel computers
Parallel Computing
Hi-index | 0.00 |
Domain-specific constraints can be exploited to implement compiler optimizations that are not otherwise feasible. Compilers for neural network learning algorithms can achieve near-optimal colocality of data and processes and near-optimal balancing of load over processors, even for dynamically irregular problems. This is impossible for general programs, but restricting programs to the neural algorithm domain allows for the exploitation of domain-specific properties. The operations performed by neural algorithms are broadcasts, reductions, and object-local operations only; the load distribution is regular with respect to the (perhaps irregular) network topology; changes of network topology occur only from time to time. A language, compilation techniques, and a compiler implementation on the MasPar MP-1 are described and quantitative results for the effects of various optimizations used in the compiler are shown. Conservative experiments with weight pruning algorithms yield performance improvements of 27 percent due to load balancing and 195 percent improvement is achieved due to data locality, both compared to unoptimized versions. Two other optimizations驴connection allocation and selecting the number of replicates驴speed programs up by about 50 percent and100 percent, respectively. This work can be viewed as a case study in exploiting domain-specific information; some of the principles presented here may apply to other domains as well.