Measuring the scalability of heterogeneous parallel systems

Authors:
Alexey Kalinov
Affiliations:
Institute for System Programming, Russian Academy of Sciences, Moscow, Russia
Venue:
PPAM'05 Proceedings of the 6th international conference on Parallel Processing and Applied Mathematics
Year:
2005

Citing 7
Cited 0

Scalability issues affecting the design of a dense linear algebra library

Journal of Parallel and Distributed Computing - Special issue on scalability of parallel algorithms and architectures
A Proposal for a Heterogeneous Cluster ScaLAPACK (Dense Linear Solvers)

IEEE Transactions on Computers
Isoefficiency: Measuring the Scalability of Parallel Algorithms and Architectures

IEEE Parallel & Distributed Technology: Systems & Technology
An efficiency and scalability model for heterogeneous clusters.

CLUSTER '01 Proceedings of the 3rd IEEE International Conference on Cluster Computing
Natural Block Data Decomposition for Heterogeneous Clusters

IPDPS '03 Proceedings of the 17th International Symposium on Parallel and Distributed Processing
Scalability Analysis of Matrix-Matrix Multiplication on Heterogeneous Clusters

ISPDC '04 Proceedings of the Third International Symposium on Parallel and Distributed Computing/Third International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Networks
Parallel Programming in C with MPI and OpenMP

Parallel Programming in C with MPI and OpenMP

Quantified Score

Hi-index	0.00

Visualization

Abstract

A parallel algorithm cannot be evaluated apart from the architecture it is implemented on. So, we define a parallel system as the combination of a parallel algorithm and a parallel architecture. The paper is devoted to the extension of well-known isoefficiency scalability metrics to heterogeneous parallel systems. Based on this extension the scalability of SUMMA (Scalable Universal Matrix Multiplication Algorithm) on parallel architecture with homogeneous communication system supporting simultaneous point-to-point communications is evaluated. Two strategies of data distribution are considered: (i) homogeneous – data are distributed between processors evenly; (ii) data are distributed between processors according to their performance. It is shown that under some assumption both strategies ensure the same scalability of heterogeneous parallel system. This theoretical result is corroborated with experiment.