Aurel Neic

Accelerating Cardiac Bidomain Simulations Using Graphics Processing Units

Anatomically realistic and biophysically detailed multiscale computer models of the heart are playing an increasingly important role in advancing our understanding of integrated cardiac function in health and disease. Such detailed simulations, however, are computationally vastly demanding, which is a limiting factor for a wider adoption of in-silico modeling. While current trends in high-performance computing (HPC) hardware promise to alleviate this problem, exploiting the potential of such architectures remains challenging since strongly scalable algorithms are necessitated to reduce execution times. Alternatively, acceleration technologies such as graphics processing units (GPUs) are being considered. While the potential of GPUs has been demonstrated in various applications, benefits in the context of bidomain simulations where large sparse linear systems have to be solved in parallel with advanced numerical techniques are less clear. In this study, the feasibility of multi-GPU bidomain simulations is demonstrated by running strong scalability benchmarks using a state-of-the-art model of rabbit ventricles. The model is spatially discretized using the finite element methods (FEM) on fully unstructured grids. The GPU code is directly derived from a large pre-existing code, the Cardiac Arrhythmia Research Package (CARP), with very minor perturbation of the code base. Overall, bidomain simulations were sped up by a factor of 11.8 to 16.3 in benchmarks running on 6-20 GPUs compared to the same number of CPU cores. To match the fastest GPU simulation which engaged 20 GPUs, 476 CPU cores were required on a national supercomputing facility.

Image-Based Personalization of Cardiac Anatomy for Coupled Electromechanical Modeling.

Computational models of cardiac electromechanics (EM) are increasingly being applied to clinical problems, with patient-specific models being generated from high fidelity imaging and used to simulate patient physiology, pathophysiology and response to treatment. Current structured meshes are limited in their ability to fully represent the detailed anatomical data available from clinical images and capture complex and varied anatomy with limited geometric accuracy. In this paper, we review the state of the art in image-based personalization of cardiac anatomy for biophysically detailed, strongly coupled EM modeling, and present our own tools for the automatic building of anatomically and structurally accurate patient-specific models. Our method relies on using high resolution unstructured meshes for discretizing both physics, electrophysiology and mechanics, in combination with efficient, strongly scalable solvers necessary to deal with the computational load imposed by the large number of degrees of freedom of these meshes. These tools permit automated anatomical model generation and strongly coupled EM simulations at an unprecedented level of anatomical and biophysical detail.

Efficient computation of electrograms and ECGs in human whole heart simulations using a reaction-eikonal model

Anatomically accurate and biophysically detailed bidomain models of the human heart have proven a powerful tool for gaining quantitative insight into the links between electrical sources in the myocardium and the concomitant current flow in the surrounding medium as they represent their relationship mechanistically based on first principles. Such models are increasingly considered as a clinical research tool with the perspective of being used, ultimately, as a complementary diagnostic modality. An important prerequisite in many clinical modeling applications is the ability of models to faithfully replicate potential maps and electrograms recorded from a given patient. However, while the personalization of electrophysiology models based on the gold standard bidomain formulation is in principle feasible, the associated computational expenses are significant, rendering their use incompatible with clinical time frames. In this study we report on the development of a novel computationally efficient reaction-eikonal (R-E) model for modeling extracellular potential maps and electrograms. Using a biventricular human electrophysiology model, which incorporates a topologically realistic His–Purkinje system (HPS), we demonstrate by comparing against a high-resolution reaction–diffusion (R–D) bidomain model that the R-E model predicts extracellular potential fields, electrograms as well as ECGs at the body surface with high fidelity and offers vast computational savings greater than three orders of magnitude. Due to their efficiency R-E models are ideally suitable for forward simulations in clinical modeling studies which attempt to personalize electrophysiological model features.

Algebraic multigrid solver on clusters of CPUs and GPUs

Solvers for elliptic partial differential equations are needed in a wide area of scientific applications. We will present a highly parallel CPU and GPU implementation of a conjugate gradient solver with an algebraic multigrid preconditioner in a package called Parallel Toolbox. The solvers operates on fully unstructured discretizations of the PDE. The algorithmic specialities are investigated with respect to many-core architectures and the code is applied to one current application. Benchmark results of computations on clusters of CPUs and GPUs will be presented. They will show that a linear equation system with 25 million unknowns can be solved in about 1 second.

Modeling Cardiac Electrophysiology at the Organ Level in the Peta FLOPS Computing Age

Despite a steep increase in available compute power, in‐silico experimentation with highly detailed models of the heart remains to be challenging due to the high computational cost involved. It is hoped that next generation high performance computing (HPC) resources lead to significant reductions in execution times to leverage a new class of in‐silico applications. However, peformance gains with these new platforms can only be achieved by engaging a much larger number of compute cores, necessitating strongly scalable numerical techniques. So far strong scalability has been demonstrated only for a moderate number of cores, orders of magnitude below the range required to achieve the desired performance boost.In this study, strong scalability of currently used techniques to solve the bidomain equations is investigated. Benchmark results suggest that scalability is limited to 512–4096 cores within the range of relevant problem sizes even when systems are carefully load‐balanced and advanced IO strategies are ...

Patient-specific modeling of left ventricular electromechanics as a driver for haemodynamic analysis

Aims Models of blood flow in the left ventricle (LV) and aorta are an important tool for analysing the interplay between LV deformation and flow patterns. Typically, image-based kinematic models describing endocardial motion are used as an input to blood flow simulations. While such models are suitable for analysing the hemodynamic status quo, they are limited in predicting the response to interventions that alter afterload conditions. Mechano-fluidic models using biophysically detailed electromechanical (EM) models have the potential to overcome this limitation, but are more costly to build and compute. We report our recent advancements in developing an automated workflow for the creation of such CFD ready kinematic models to serve as drivers of blood flow simulations. Methods and results EM models of the LV and aortic root were created for four pediatric patients treated for either aortic coarctation or aortic valve disease. Using MRI, ECG and invasive pressure recordings, anatomy as well as electrophysiological, mechanical and circulatory model components were personalized. Results The implemented modeling pipeline was highly automated and allowed model construction and execution of simulations of a patient’s heartbeat within 1 day. All models reproduced clinical data with acceptable accuracy. Conclusion Using the developed modeling workflow, the use of EM LV models as driver of fluid flow simulations is becoming feasible. While EM models are costly to construct, they constitute an important and nontrivial step towards fully coupled electro-mechano-fluidic (EMF) models and show promise as a tool for predicting the response to interventions which affect afterload conditions.

Stepping into Fully GPU Accelerated Biomedical Applications

We present ideas and first results on a GPU acceleration of a non-linear solver embedded into the biomedical application code CARP. The linear system solvers have been transferred already in the past and so we concentrate on how to extend the GPU acceleration to larger portions of the code. The finite element assembling of stiffness and mass matrices takes at least 50 % of the CPU time and therefore we investigate this process for the bidomain equations but with focus on later use in non-linear and/or time-dependent problems. The CUDA code for matrix calculation and assembling is faster by a factor up to \(90\) compared to a single CPU core. The routines were integrated to CARP’s main code and they are already used to assemble the FE matrices of the bidomain model. Further performance studies are still required for the bidomain-mechanics model.

Universal ventricular coordinates: A generic framework for describing position within the heart and transferring data

&NA; Being able to map a particular set of cardiac ventricles to a generic topologically equivalent representation has many applications, including facilitating comparison of different hearts, as well as mapping quantities and structures of interest between them. In this paper we describe Universal Ventricular Coordinates (UVC), which can be used to describe position within any biventricular heart. UVC comprise four unique coordinates that we have chosen to be intuitive, well defined, and relevant for physiological descriptions. We describe how to determine these coordinates for any volumetric mesh by illustrating how to properly assign boundary conditions and utilize solutions to Laplaces equation. Using UVC, we transferred scalar, vector, and tensor data between four unstructured ventricular meshes from three different species. Performing the mappings was very fast, on the order of a few minutes, since mesh nodes were searched in a KD tree. Distance errors in mapping mesh nodes back and forth between meshes were less than the size of an element. Analytically derived fiber directions were also mapped across meshes and compared, showing < 5° difference over most of the ventricles. The ability to transfer gradients was also demonstrated. Topologically variable structures, like papillary muscles, required further definition outside of the UVC framework. In conclusion, UVC can aid in transferring many types of data between different biventricular geometries.

Towards a Computational Framework for Modeling the Impact of Aortic Coarctations upon Left Ventricular Load

Computational fluid dynamics (CFD) models of blood flow in the left ventricle (LV) and aorta are important tools for analyzing the mechanistic links between myocardial deformation and flow patterns. Typically, the use of image-based kinematic CFD models prevails in applications such as predicting the acute response to interventions which alter LV afterload conditions. However, such models are limited in their ability to analyze any impacts upon LV load or key biomarkers known to be implicated in driving remodeling processes as LV function is not accounted for in a mechanistic sense. This study addresses these limitations by reporting on progress made toward a novel electro-mechano-fluidic (EMF) model that represents the entire physics of LV electromechanics (EM) based on first principles. A biophysically detailed finite element (FE) model of LV EM was coupled with a FE-based CFD solver for moving domains using an arbitrary Eulerian-Lagrangian (ALE) formulation. Two clinical cases of patients suffering from aortic coarctations (CoA) were built and parameterized based on clinical data under pre-treatment conditions. For one patient case simulations under post-treatment conditions after geometric repair of CoA by a virtual stenting procedure were compared against pre-treatment results. Numerical stability of the approach was demonstrated by analyzing mesh quality and solver performance under the significantly large deformations of the LV blood pool. Further, computational tractability and compatibility with clinical time scales were investigated by performing strong scaling benchmarks up to 1536 compute cores. The overall cost of the entire workflow for building, fitting and executing EMF simulations was comparable to those reported for image-based kinematic models, suggesting that EMF models show potential of evolving into a viable clinical research tool.