Three-level hybrid vs. flat MPI on the Earth Simulator: Parallel iterative solvers for finite-element method

Authors:
Kengo Nakajima
Affiliations:
The 21st Century Earth Science COE Program, Department of Earth and Planetary Science, The University of Tokyo, Japan
Venue:
Applied Numerical Mathematics - 6th IMACS International symposium on iterative methods in scientific computing
Year:
2005

Citing 8
Cited 4

Domain decomposition: parallel multilevel methods for elliptic partial differential equations

Domain decomposition: parallel multilevel methods for elliptic partial differential equations
MPI versus MPI+OpenMP on IBM SP for the NAS benchmarks

Proceedings of the 2000 ACM/IEEE conference on Supercomputing
Solving Linear Systems on Vector and Shared Memory Computers

Solving Linear Systems on Vector and Shared Memory Computers
Effects of Ordering Strategies and Programming Paradigms on Sparse Matrix Computations

SIAM Review
GeoFEM: High Performance Parallel FEM for Solid Earth

HPCN Europe '99 Proceedings of the 7th International Conference on High-Performance Computing and Networking
Communication Bandwidth of Parallel Programming Models on Hybrid Architectures

ISHPC '02 Proceedings of the 4th International Symposium on High Performance Computing
Iterative Methods for Sparse Linear Systems

Iterative Methods for Sparse Linear Systems
Parallel Iterative Solvers for Unstructured Grids Using an OpenMP/MPI Hybrid Programming Model for the GeoFEM Platform on SMP Cluster Architectures

ISHPC '02 Proceedings of the 4th International Symposium on High Performance Computing

Parallel programming models for finite-element method using preconditioned iterative solvers with multicolor ordering on various types of SMP cluster supercomputers

HPCASIA '05 Proceedings of the Eighth International Conference on High-Performance Computing in Asia-Pacific Region
The impact of parallel programming models on the performance of iterative linear solvers for finite element applications

VECPAR'06 Proceedings of the 7th international conference on High performance computing for computational science
Optimizing the advanced accelerator simulation framework synergia using OpenMP

IWOMP'12 Proceedings of the 8th international conference on OpenMP in a Heterogeneous World
Multi-level parallelism for incompressible flow computations on GPU clusters

Parallel Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

An efficient parallel iterative method for finite element method has been developed for symmetric multiprocessor (SMP) cluster architectures with vector processors such as the Earth Simulator. The method is based on a three-level hybrid parallel programming model, including message passing for inter-SMP node communication, loop directives by OpenMP for intra-SMP node parallelization and vectorization for each processing element (PE). Simple 3D linear elastic problems with more than 2.2x10^9 DOF have been solved using 3x3 block ICCG(0) method with additive Schwarz domain decomposition and PDJDS/CM-RCM reordering on 176 nodes of the Earth Simulator, achieving performance of 3.80 TFLOPS. The PDJDS/CM-RCM reordering method provides excellent vector and parallel performance in SMP nodes. A three-level hybrid parallel programming model outperforms flat MPI in the problems involving large numbers of SMP nodes.