Exploiting processor groups to extend scalability of the GA shared memory programming model
Proceedings of the 2nd conference on Computing frontiers
| Hi-index | 0.00 |
Realistic simulations of flow past a flexible cylinder subject to vortex-induced vibrations require a large number of Fourier modes along the cylinder span and high resolutions in the streamwise and cross-flow directions. Parallel computations employing a single-level parallelism for this type of problems have clear performance limitations that prevent effective scaling tothe large processor count on modern supercomputers. In this paper we present two multilevel parallel paradigms based on MPI/MPI and MPI/OpenMP for high-order CFD methods within the spectral element framework and compare their performance. In the MPI/MPI model, weemploy MPI process groups/communicators to decompose the flow domain and MPI processes into different levels. In the MPI/OpenMP model, we employ multiple OpenMP threads to split the workload within the sub-domain and take a coarse-grain approach thatsignificantly reduces the OpenMP synchronizations. For identical configurations the MPI/MPI model is observed to be generally more efficient. However, for dynamic prefinement the MPI/OpenMP approach is more effective. Because a greatly reduced number of processes are involved in the communications at each level, these multilevel parallel paradigms reduce the network latency overhead and enable the applications to scale to a large number of processors more efficiently.