A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the Earth Simulator

Authors:
Dimitri Komatitsch;Seiji Tsuboi;Chen Ji;Jeroen Tromp
Affiliations:
Caltech, Pasadena, California;IFREE, JAMSTEC, Yokohama, Japan;Caltech, Pasadena, California;Caltech, Pasadena, California
Venue:
Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Year:
2003

Citing 2
Cited 24

The “cubed sphere”: a new method for the solution of partial differential equations in spherical geometry

Journal of Computational Physics
A 26.58 Tflops global atmospheric simulation with the spectral transform method on the Earth Simulator

Proceedings of the 2002 ACM/IEEE conference on Supercomputing

A 15.2 TFlops Simulation of Geodynamo on the Earth Simulator

Proceedings of the 2004 ACM/IEEE conference on Supercomputing
Guest editorial: the earth simulator

Parallel Computing
High-Resolution Mesh Convergence Properties and Parallel Efficiency of a Spectral Element Atmospheric Dynamical Core

International Journal of High Performance Computing Applications
16.447 TFlops and 159-Billion-dimensional Exact-diagonalization for Trapped Fermion-Hubbard Model on the Earth Simulator

SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
Efficient query processing on unstructured tetrahedral meshes

Proceedings of the 2006 ACM SIGMOD international conference on Management of data
High-performance computing for exact numerical approaches to quantum many-body problems on the earth simulator

Proceedings of the 2006 ACM/IEEE conference on Supercomputing
High-frequency simulations of global seismic wave propagation using SPECFEM3D_GLOBE on 62K processors

Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Simulation of Seismic Wave Propagation in an Asteroid Based upon an Unstructured MPI Spectral-Element Method: Blocking and Non-blocking Communication Strategies

High Performance Computing for Computational Science - VECPAR 2008
A Simulation of Seismic Wave Propagation at High Resolution in the Inner Core of the Earth on 2166 Processors of MareNostrum

High Performance Computing for Computational Science - VECPAR 2008
Porting a high-order finite-element earthquake modeling application to NVIDIA graphics cards using CUDA

Journal of Parallel and Distributed Computing
A Non-oscillatory Advection Operator for the Compatible Spectral Element Method

ICCS 2009 Proceedings of the 9th International Conference on Computational Science
Understanding earthquakes with advanced visualization

ACM SIGGRAPH Computer Graphics - Visual Research, Evaluation and Assessment in the Age of Computer Graphics
Progress towards petascale applications in biology: status in 2006

Euro-Par'06 Proceedings of the CoreGRID 2006, UNICORE Summit 2006, Petascale Computational Biology and Bioinformatics conference on Parallel processing
16.14 TFLOPS eigenvalue solver on the earth simulator: exact diagonalization for ultra largescale hamiltonian matrix

ISHPC'05/ALPS'06 Proceedings of the 6th international symposium on high-performance computing and 1st international conference on Advanced low power systems
Computations of global seismic wave propagation in three dimensional earth model

ISHPC'05/ALPS'06 Proceedings of the 6th international symposium on high-performance computing and 1st international conference on Advanced low power systems
High-performance finite-element simulations of seismic wave propagation in three-dimensional nonlinear inelastic geological media

Parallel Computing
High-order finite-element seismic wave propagation modeling with MPI on a large GPU cluster

Journal of Computational Physics
A survey of the practice of computational science

State of the Practice Reports
FTI: high performance fault tolerance interface for hybrid systems

Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
Extreme-scale UQ for Bayesian inverse problems governed by PDEs

SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Forward and adjoint simulations of seismic wave propagation on emerging large-scale GPU architectures

SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Energy efficiency vs. performance of the numerical solution of PDEs: An application study on a low-power ARM-based cluster

Journal of Computational Physics
Accelerating spatial range queries

Proceedings of the 16th International Conference on Extending Database Technology
Data-driven neuroscience: enabling breakthroughs via innovative data management

Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data

Quantified Score

Hi-index	0.01

Visualization

Abstract

We use 1944 processors of the Earth Simulator to model seismic wave propagation resulting from large earthquakes. Simulations are conducted based upon the spectral-element method, a high-degree finite-element technique with an exactly diagonal mass matrix. We use a very large mesh with 5.5 billion grid points (14.6 billion degrees of freedom). We include the full complexity of the Earth, i.e., a three-dimensional wave-speed and density structure, a 3-D crustal model, ellipticity as well as topography and bathymetry. A total of 2.5 terabytes of memory is needed. Our implementation is purely based upon MPI, with loop vectorization on each processor. We obtain an excellent vectorization ratio of 99.3%, and we reach a performance of 5 teraflops (30% of the peak performance) on 38% of the machine. The very high resolution of the mesh allows us to perform fully three-dimensional calculations at seismic periods as low as 5 seconds.