Václav Hapla

Total FETI domain decomposition method and its massively parallel implementation

We describe an efficient massively parallel implementation of our variant of the FETI type domain decomposition method called Total FETI with a lumped preconditioner. A special attention is paid to the discussion of several variants of parallelization of the action of the projections to the natural coarse grid and to the effective regularization of the stiffness matrices of the subdomains. Both numerical and parallel scalability of the proposed TFETI method are demonstrated on a 2D elastostatic benchmark up to 314,505,600 unknowns and 4800cores. The results are also important for implementation of scalable algorithms for the solution of nonlinear contact problems of elasticity by TFETI based domain decomposition method.

TFETI coarse space projectors parallelization strategies

This paper deals with an analysis of various parallelization strategies for the TFETI algorithm. The data distributions and the most relevant actions are discussed, especially those concerning coarse problem. Being a part of the coarse space projector, it couples all the subdomains computations and accelerates convergence. Its direct solution is more robust but complicates the massively parallel implementation. Presented numerical results confirm high parallel scalability of the coarse problem solution if the dual constraint matrix is distributed and then orthonormalized in parallel. Benchmarks were implemented using PETSc parallelization library and run on HECToR service at EPCC in Edinburgh.

Solving Contact Mechanics Problems with PERMON

PERMON makes use of theoretical results in quadratic programming algorithms and domain decomposition methods. It is built on top of the PETSc framework for numerical computations. This paper describes its fundamental packages and shows their applications. We focus here on contact problems of mechanics decomposed by means of a FETI-type non-overlapping domain decomposition method. These problems lead to inequality constrained quadratic programming problems that can be solved by our PermonQP package.

Parallel Implementation of the FETI DDM Constraint Matrix on Top of PETSc for the PermonFLLOP Package

This paper deals with implementation of the FETI non-overlapping domain decomposition method within our new software toolbox PERMON, combining quadratic programming algorithms and domain decomposition methods. It is built on top of the PETSc framework for numerical computations. Particularly, we focus on parallel implementation of the matrix which manages connectivity between subdomains within the FETI method. We present a basic idea of our approach based on processing local and global numberings of the degrees of freedom on subdomain interfaces.

Massively parallel solution of elastoplasticity problems with tens of millions of unknowns using PermonCube and FLLOP packages

In this paper we are presenting our PermonCube and FLLOP packages, and their use for massively parallel solution of elastoplasticity problems. PermonCube provides simple cubical meshes, partitioned in a non-overlapping manner. By means of finite element method it assembles all linear algebra objects required for solution of the physical problem. Two chosen nonlinear material models are presented, and a solving strategy based on the Newtons method is briefly discussed. PermonCube uses our FLLOP library as a linear system solver. FLLOP is able to solve problems decomposed in a non-overlapping manner using domain decomposition methods of the FETI type. It extends PETSc (Portable, Extensible Toolkit for Scientific Computation). In the last section, large-scale numerical experiments with problem size up to 60 million of degrees of freedom are presented.

Implementation of the efficient communication layer for the highly parallel total FETI and hybrid total FETI solvers

Implementation, performance, and scalability results of communication layer for Total FETI and Hybrid Total FETI solver.In HTFETI several neighboring subdomains aggregated into clusters. This reduces the size of coarse problem and improves scalability.Optimization of nearest neighbor communication - global gluing matrix.Implementation of communication hiding and avoiding techniques inside the communication layerBenchmarks - elastic 3D cube up to 1.6 billion DOF and realistic car engine benchmark.Large test executed on Total FETI to see the real potential of communication layer on smaller clusters. This paper describes the implementation, performance, and scalability of our communication layer developed for Total FETI (TFETI) and Hybrid Total FETI (HTFETI) solvers. HTFETI is based on our variant of the Finite Element Tearing and Interconnecting (FETI) type domain decomposition method. In this approach a small number of neighboring subdomains is aggregated into clusters, which results in a smaller coarse problem. To solve the original problem TFETI method is applied twice: to the clusters and then to the subdomains in each cluster.The current implementation of the solver is focused on the performance optimization of the main CG iteration loop, including: implementation of communication hiding and avoiding techniques for global communications; optimization of the nearest neighbor communication - multiplication with a global gluing matrix; and optimization of the parallel CG algorithm to iterate over local Lagrange multipliers only.The performance is demonstrated on a linear elasticity 3D cube and real world benchmarks.

Efficient lifetime estimation techniques for general multiaxial loading

In this paper, we discuss and present our progress toward a project, which is focused on fatigue life prediction under multiaxial loading in the domain of low-cycle fatigue, i.e. cases, where the plasticity cannot be neglected. First, the elastic-plastic solution in the finite element analysis is enhanced and verified on own experiments. Second, the method by Jiang describing the instantaneous damage increase by analyses of load time by time, is in implementation phase. In addition, simplified routines for conversion of elastic stresses-strains to elastic-plastic ones as proposed by Firat and Ye et.al. are evaluated on the basis of data gathered from external sources. In order to produce high quality complex analyses, which could be feasible in an acceptable time, and allow the period for next analyses of results to be expanded; the core of PragTic fatigue solver used for all fatigue computations are being re-implemented to get the fully parallelized scalable solution.

FLLOP: A Massively Parallel Solver Combining FETI Domain Decomposition Method and Quadratic Programming

FLLOP (FETI Light Layer On top of PETSc) is a novel solver for linear systems and more general quadratic programming problems. It connects quadratic programming algorithms with FETI domain decomposition methods that bring high parallel scalability and at the same time high accuracy and robustness needed in engineering applications. According to its name, FLLOP forms an extension of the renowned PETSc software framework for parallel numerical solution of partial differential equations. It can be thus adopted by PETSc users with relative ease. The above-mentioned facts makes FLLOP a unique project. It now forms a solver part of our emerging PERMON toolbox for Parallel, Efficient, Robust, Modular, Object-oriented, Numerical simulations. We introduce one more PERMON tool, Permon Cube, which serves as a benchmark generation tool for FLLOP in massively parallel environment.

The impact of enabling multiple subdomains per MPI process in the TFETI domain decomposition method

Abstract The paper deals with handling multiple subdomains per computational core in the PERMON toolbox, namely in the PermonFLLOP module, to fully exploit the potential of the Total Finite Element Tearing and Interconnecting (TFETI) domain decomposition method (DDM). Most authors researching FETI methods present weak parallel scalability with one subdomain assigned to each computational core, and call it just parallel scalability. Here we present an extension showing the data of more than one subdomain being held by each MPI process. Numerical experiments demonstrate the theoretically supported fact that for the given problem size and number of processors, the increased number of subdomains leads to better conditioning of the system operator, and hence faster convergence. Moreover, numerical, memory, strong parallel, and weak parallel scalability is reported, and optimal numbers of subdomains per core are examined. Finally, new PETSc matrix types dealing with the aforementioned extension are introduced.

PERMON software toolbox as solver of contact problems in mechanics

PERMON toolbox makes use of theoretical results in discretization techniques, quadratic programming algorithms, and domain decomposition methods. It is built on top of the PETSc framework for numerical computations. This paper describes its packages, and shows case of their application for a contact problem of linear elasticity decomposed by TFETI non-overlapping domain decomposition method. We end with the inequality constrained quadratic programming problems that can be solved by our PermonQP package with the PermonIneq extension.