Dimitri Komatitsch

An unsplit convolutional Perfectly Matched Layer improved at grazing incidence for the seismic wave equation

The perfectly matched layer (PML) absorbing boundary condition has proven to be very efficient from a numerical point of view for the elastic wave equation to absorb both body waves with nongrazing incidence and surface waves. However, at grazing incidence the classical discrete PML method suffers from large spurious reflections that make it less efficient for instance in the case of very thin mesh slices, in the case of sources located close to the edge of the mesh, and/or in the case of receivers located at very large offset. We demonstrate how to improve the PML at grazing incidence for the differential seismic wave equation based on an unsplit convolution technique. The improved PML has a cost that is similar in terms of memory storage to that of the classical PML. We illustrate the efficiency of this improved convolutional PML based on numerical benchmarks using a finite-difference method on a thin mesh slice for an isotropic material and show that results are significantly improved compared with the classical PML technique. We also show that, as the classical PML, the convolutional technique is intrinsically unstable in the case of some anisotropic materials.

Simulations of Ground Motion in the Los Angeles Basin Based upon the Spectral-Element Method

We use the spectral-element method to simulate ground motion generated by two recent and well-recorded small earthquakes in the Los Angeles basin. Simulations are performed using a new sedimentary basin model that is constrained by hundreds of petroleum-industry well logs and more than 20,000 km of seismic reflection profiles. The numerical simulations account for 3D variations of seismic-wave speeds and density, topography and bathymetry, and attenuation. Simulations for the 9 September 2001 M_w 4.2 Hollywood earthquake and the 3 September 2002 M_w 4.2 Yorba Linda earthquake demonstrate that the combination of a detailed sedimentary basin model and an accurate numerical technique facilitates the simulation of ground motion at periods of 2 sec and longer inside the basin model and 6 sec and longer in the regional model. Peak ground displacement, velocity, and acceleration maps illustrate that significant amplification occurs in the basin.

High-order finite-element seismic wave propagation modeling with MPI on a large GPU cluster

We implement a high-order finite-element application, which performs the numerical simulation of seismic wave propagation resulting for instance from earthquakes at the scale of a continent or from active seismic acquisition experiments in the oil industry, on a large cluster of NVIDIA Tesla graphics cards using the CUDA programming environment and non-blocking message passing based on MPI. Contrary to many finite-element implementations, ours is implemented successfully in single precision, maximizing the performance of current generation GPUs. We discuss the implementation and optimization of the code and compare it to an existing very optimized implementation in C language and MPI on a classical cluster of CPU nodes. We use mesh coloring to efficiently handle summation operations over degrees of freedom on an unstructured mesh, and non-blocking MPI messages in order to overlap the communications across the network and the data transfer to and from the device via PCIe with calculations on the GPU. We perform a number of numerical tests to validate the single-precision CUDA and MPI implementation and assess its accuracy. We then analyze performance measurements and depending on how the problem is mapped to the reference CPU cluster, we obtain a speedup of 20x or 12x.

Simulation of anisotropic wave propagation based upon a spectral element method

We introduce a numerical approach for modeling elastic wave propagation in 2-D and 3-D fully anisotropic media based upon a spectral element method. The technique solves a weak formulation of the wave equation, which is discretized using a high-order polynomial representation on a finite element mesh. For isotropic media, the spectral element method is known for its high degree of accuracy, its ability to handle complex model geometries, and its low computational cost. We show that the method can be extended to fully anisotropic media. The mass matrix obtained is diagonal by construction, which leads to a very efficient fully explicit solver. We demonstrate the accuracy of the method by comparing it against a known analytical solution for a 2-D transversely isotropic test case, and by comparing its predictions against those based upon a finite difference method for a 2-D heterogeneous, anisotropic medium. We show its generality and its flexibility by modeling wave propagation in a 3-D transversely isotropic medium with a symmetry axis tilted relative to the axes of the grid.

Spectral Element Analysis in Seismology

We present a review of the application of the spectral-element method to regional and global seismology. This technique is a high-order variational method that allows one to compute accurate synthetic seismograms in three-dimensional heterogeneous Earth models with deformed geometry. We first recall the strong and weak forms of the seismic wave equation with a particular emphasis set on fluid regions. We then discuss in detail how the conditions that hold on the boundaries, including coupling boundaries, are honored. We briefly outline the spectral-element discretization procedure and present the time-marching algorithm that makes use of the diagonal structure of the mass matrix. We show examples that illustrate the capabilities of the method and its interest in the context of the computation of three-dimensional synthetic seismograms.

Porting a high-order finite-element earthquake modeling application to NVIDIA graphics cards using CUDA

We port a high-order finite-element application that performs the numerical simulation of seismic wave propagation resulting from earthquakes in the Earth on NVIDIA GeForce 8800 GTX and GTX 280 graphics cards using CUDA. This application runs in single precision and is therefore a good candidate for implementation on current GPU hardware, which either does not support double precision or supports it but at the cost of reduced performance. We discuss and compare two implementations of the code: one that has maximum efficiency but is limited to the memory size of the card, and one that can handle larger problems but that is less efficient. We use a coloring scheme to handle efficiently summation operations over nodes on a topology with variable valence. We perform several numerical tests and performance measurements and show that in the best case we obtain a speedup of 25.

Wave propagation near a fluid‐solid interface: A spectral‐element approach

We introduce a spectral-element method for modeling wave propagation in media with both fluid (acoustic) and solid (elastic) regions, as for instance in offshore seismic experiments. The problem is formulated in terms of displacement in elastic regions and a velocity potential in acoustic regions. Matching between domains is implemented based upon an interface integral in the framework of an explicit prediction-multicorrection staggered time scheme. The formulation results in a mass matrix that is diagonal by construction. The scheme exhibits high accuracy for a 2-D test case with known analytical solution. The method is robust in the case of strong topography at the fluid-solid interface and is a good alternative to classical techniques, such as finite differencing.

Spectral-Element Moment Tensor Inversions for Earthquakes in Southern California

We have developed and implemented an automated moment tensor inversion procedure to determine source parameters for southern California earthquakes. The method is based upon spectral-element simulations of regional seismic wave propagation in an integrated 3D southern California velocity model. Sensitivity to source parameters is determined by numerically calculating the Frechet derivatives required for the moment tensor inversion. We minimize a waveform misfit function, and allow limited time shifts between data and corresponding synthetics to accommodate additional 3D heterogeneity not included in our model. The technique is applied to three recent southern California earthquakes: the 9 September 2001, M_L 4.2 Hollywood event, the 22 February 2003, M_L 5.4 Big Bear event, and the 14 December 2001, M_L 4.0 Diamond Bar event. Using about half of the available three-component data at periods of 6 sec and longer, we obtain focal mechanisms, depths, and moment magnitudes that are generally in good agreement with estimates based upon traditional body-wave and surface-wave inversions.

The Spectral‐Element Method in Seismology

We present the main properties of the spectral-element method, which is well suited for numerical calculations of synthetic seismograms for three-dimensional Earth models. The technique is based upon a weak formulation of the equations of motion and combines the flexibility of a finite-element method with the accuracy of a pseudospectral method. The mesh is composed of hexahedral elements and honors the main discontinuities in the Earth model. The displacement vector is expressed in each element in terms of high-degree Lagrange interpolants, and integrals are computed based upon Gauss-Lobatto-Legendre quadrature, which leads to an exactly diagonal mass matrix and therefore drastically simplifies the algorithm. We use a fluid-solid coupling formulation that does not require iterations at the core-mantle or inner-core boundaries. The method is efficiently implemented on parallel computers with distributed memory based upon a message-passing methodology. We present two large-scale simulations for a realistic three-dimensional Earth model computed on the Japanese Earth Simulator at periods of 5 s and longer.

A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the Earth Simulator

We use 1944 processors of the Earth Simulator to model seismic wave propagation resulting from large earthquakes. Simulations are conducted based upon the spectral-element method, a high-degree finite-element technique with an exactly diagonal mass matrix. We use a very large mesh with 5.5 billion grid points (14.6 billion degrees of freedom). We include the full complexity of the Earth, i.e., a three-dimensional wave-speed and density structure, a 3-D crustal model, ellipticity as well as topography and bathymetry. A total of 2.5 terabytes of memory is needed. Our implementation is purely based upon MPI, with loop vectorization on each processor. We obtain an excellent vectorization ratio of 99.3%, and we reach a performance of 5 teraflops (30% of the peak performance) on 38% of the machine. The very high resolution of the mesh allows us to perform fully three-dimensional calculations at seismic periods as low as 5 seconds.