Hiding Communication Latency with Non-SPMD, Graph-Based Execution

Authors:
Jacob Sorensen;Scott B. Baden
Affiliations:
Department of Computer Science and Engineering, University of California, San Diego, La Jolla, USA CA 92093-0404;Department of Computer Science and Engineering, University of California, San Diego, La Jolla, USA CA 92093-0404
Venue:
ICCS '09 Proceedings of the 9th International Conference on Computational Science: Part I
Year:
2009

Citing 15
Cited 1

The Manchester prototype dataflow computer

Communications of the ACM - Special section on computer architecture
Executing a Program on the MIT Tagged-Token Dataflow Architecture

IEEE Transactions on Computers
A parallel hashed Oct-Tree N-body algorithm

Proceedings of the 1993 ACM/IEEE conference on Supercomputing
Cilk: an efficient multithreaded runtime system

PPOPP '95 Proceedings of the fifth ACM SIGPLAN symposium on Principles and practice of parallel programming
Portable run-time support for dynamic object-oriented parallel processing

ACM Transactions on Computer Systems (TOCS)
A data-localization compilation scheme using partial-static task assignment for Fortran coarse-grain parallel processing

Parallel Computing - Special issues on languages and compilers for parallel computers
SMARTS: exploiting temporal locality and parallelism through vertical execution

ICS '99 Proceedings of the 13th international conference on Supercomputing
Performance Tradeoffs in Multi-tier Formulation of a Finite Difference Method

ICCS '01 Proceedings of the International Conference on Computational Sciences-Part I
NAMD: biomolecular simulation on thousands of processors

Proceedings of the 2002 ACM/IEEE conference on Supercomputing
Asynchronous programming with Tarragon

Proceedings of the 2006 ACM/IEEE conference on Supercomputing
Optimizing communication overlap for high-speed networks

Proceedings of the 12th ACM SIGPLAN symposium on Principles and practice of parallel programming
Charisma: orchestrating migratable parallel objects

Proceedings of the 16th international symposium on High performance distributed computing
Parallel Processing with Large-Grain Data Flow Techniques

Computer
Data Flow Supercomputers

Computer
Multi-threading and one-sided communication in parallel LU factorization

Proceedings of the 2007 ACM/IEEE conference on Supercomputing

Bamboo: translating MPI applications to a latency-tolerant, data-driven form

SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

Reformulating an algorithm to mask communication delays is crucial in maintaining scalability, but traditional solutions embed the overlap strategy into the application. We present an alternative approach based on dataflow, that factors the overlap strategy out of the application. Using this approach we are able to reduce communication delays, meeting and in many cases exceeding performance obtained with traditional hand coded applications.