Spectral element—Fourier methods for incompressible turbulent flows
ICOSAHOM '89 Proceedings of the conference on Spectral and high order methods for partial differential equations
The GENESIS distributed-memory benchmarks
Computer benchmarks
Unstructured spectral element methods for simulation of turbulent flows
Journal of Computational Physics
An Architecture for Optimal All-to-All Personalized Communication
An Architecture for Optimal All-to-All Personalized Communication
LAPACK Working Note 65: Parallel Matrix Transpose Algorithms on Distributed Memory Concurrent Computers
Message-Passing Performance of Various Computers
Message-Passing Performance of Various Computers
Dual-level parallelism for deterministic and stochastic CFD problems
Proceedings of the 2002 ACM/IEEE conference on Supercomputing
A Scalable Interconnection Network Architecture for Petaflops Computing
The Journal of Supercomputing
Journal of Computational Physics
| Hi-index | 0.00 |
In this paper we analyze communication patterns in the parallel three-dimensional Navier-Stokes solver Prism, and present performance results on the IBM SP2, the Cray T3D and the SGI Power Challenge XL. Prism is used for direct numerical simulation of turbulence in non-separable and multiply-connected domains. The numerical method used in the solver is based on mixed spectral element-Fourier expansions in (x-y) planes and z-direction, respectively. Each (or a group) of Fourier modes is computed on a separate processor as the linear contributions (Helmholtz solves) are completely uncoupled in the incompressible Navier-Stokes equations; coupling is obtained via the nonlinear contributions (convective terms). The transfer of data between physical and Fourier space requires a series of complete exchange operations, which dominate the communication cost for small number of processors. As the number of processors increases, global reduction and gather operations become important while complete exchange becomes more latency dominated. Predictive models for these communication operations are proposed and tested against measurements. A relatively large variation in communication timings per iteration is observed in simulations and quantified in terms of specific operations. A number of improvements are proposed that could significantly reduce the communications overhead with increasing numbers of processors, and {\em generic} predictive maps are developed for the complete exchange operation, which remains the fundamental communication in Prism. Results presented in this paper are representative of a wider class of parallel spectral and finite element codes for computational mechanics which require similar communication operations.