Jazmin Aguado-Sierra

Alya: Multiphysics engineering simulation toward exascale

Alya is a multi-physics simulation code developed at Barcelona Supercomputing Center (BSC). From its inception Alya code is designed using advanced High Performance Computing programming techniques to solve coupled problems on supercomputers efficiently. The target domain is engineering, with all its particular features: complex geometries and unstructured meshes, coupled multi-physics with exotic coupling schemes and physical models, ill-posed problems, flexibility needs for rapidly including new models, etc. Since its beginnings in 2004, Alya has scaled well in an increasing number of processors when solving single-physics problems such as fluid mechanics, solid mechanics, acoustics, etc. Over time, we have made a concerted effort to maintain and even improve scalability for multi-physics problems. This poses challenges on multiple fronts, including: numerical models, parallel implementation, physical coupling models, algorithms and solution schemes, meshing process, etc. In this paper, we introduce Alyas main features and focus particularly on its solvers. We present Alyas performance up to 100.000 processors in Blue Waters, the NCSA supercomputer with selected multi-physics tests that are representative of the engineering world. The tests are incompressible flow in a human respiratory system, low Mach combustion problem in a kiln furnace, and coupled electro-mechanical contraction of the heart. We show scalability plots for all cases and discuss all aspects of such simulations, including solver convergence.

Alya Red CCM: HPC-Based Cardiac Computational Modelling

This paper describes Alya Red CCM, a cardiac computational modelling tool for supercomputers. It is based on Alya, a parallel simulation code for multiphysics and multiscale problems, which can deal with all the complexity of biological systems simulations. The final goal is to simulate the pumping action of the heart: the model includes the electrical propagation, the mechanical contraction and relaxation and the blood flow in the heart cavities and main vessels. All sub-problems are treated as fully transient and solved in a staggered fashion. Electrophysiology and mechanical deformation are solved on the same mesh, with no interpolation. Fluid flow is simulated on a moving mesh using an Arbitrary Lagrangian-Eulerian (ALE) strategy, being the mesh deformation computed through an anisotropic Laplacian equation. The parallel strategy is based on an automatic mesh partition using Metis and MPI tasks. When required and in order to take profit of multicore clusters, an additional OpenMP parallelization layer is added. The paper describes the tool and its different parts. Alya’s flexibility allows to easily program a large variety of physiological models for each of the sub-problems, including the mutual coupling. This flexibility, added to the parallel efficiency to solve multiphysics problems in complex geometries render Alya Red CCM a well suited tool for cardiac biomedical research at either industrial or academic environments.

Evaluating the roles of detailed endocardial structures on right ventricular haemodynamics by means of CFD simulations

Computational modelling plays an important role in right ventricular (RV) haemodynamic analysis. However, current approaches use smoothed ventricular anatomies. The aim of this study is to characterise RV haemodynamics including detailed endocardial structures like trabeculae, moderator band, and papillary muscles. Four paired detailed and smoothed RV endocardium models (2 male and 2 female) were reconstructed from ex vivo human hearts high-resolution magnetic resonance images. Detailed models include structures with ≥1 mm2 cross-sectional area. Haemodynamic characterisation was done by computational fluid dynamics simulations with steady and transient inflows, using high-performance computing. The differences between the flows in smoothed and detailed models were assessed using Q-criterion for vorticity quantification, the pressure drop between inlet and outlet, and the wall shear stress. Results demonstrated that detailed endocardial structures increase the degree of intra-ventricular pressure drop, decrease the wall shear stress, and disrupt the dominant vortex creating secondary small vortices. Increasingly turbulent blood flow was observed in the detailed RVs. Female RVs were less trabeculated and presented lower pressure drops than the males. In conclusion, neglecting endocardial structures in RV haemodynamic models may lead to inaccurate conclusions about the pressures, stresses, and blood flow behaviour in the cavity.

Left Ventricular Trabeculations Decrease the Wall Shear Stress and Increase the Intra-Ventricular Pressure Drop in CFD Simulations

The aim of the present study is to characterize the hemodynamics of left ventricular (LV) geometries to examine the impact of trabeculae and papillary muscles (PMs) on blood flow using high performance computing (HPC). Five pairs of detailed and smoothed LV endocardium models were reconstructed from high-resolution magnetic resonance images (MRI) of ex-vivo human hearts. The detailed model of one LV pair is characterized only by the PMs and few big trabeculae, to represent state of art level of endocardial detail. The other four detailed models obtained include instead endocardial structures measuring ≥1 mm2 in cross-sectional area. The geometrical characterizations were done using computational fluid dynamics (CFD) simulations with rigid walls and both constant and transient flow inputs on the detailed and smoothed models for comparison. These simulations do not represent a clinical or physiological scenario, but a characterization of the interaction of endocardial structures with blood flow. Steady flow simulations were employed to quantify the pressure drop between the inlet and the outlet of the LVs and the wall shear stress (WSS). Coherent structures were analyzed using the Q-criterion for both constant and transient flow inputs. Our results show that trabeculae and PMs increase the intra-ventricular pressure drop, reduce the WSS and disrupt the dominant single vortex, usually present in the smoothed-endocardium models, generating secondary small vortices. Given that obtaining high resolution anatomical detail is challenging in-vivo, we propose that the effect of trabeculations can be incorporated into smoothed ventricular geometries by adding a porous layer along the LV endocardial wall. Results show that a porous layer of a thickness of 1.2·10−2 m with a porosity of 20 kg/m2 on the smoothed-endocardium ventricle models approximates the pressure drops, vorticities and WSS observed in the detailed models.

Implications of bipolar voltage mapping and magnetic resonance imaging resolution in biventricular scar characterization after myocardial infarction

Aims We aimed to study the differences in biventricular scar characterization using bipolar voltage mapping compared with state-of-the-art in vivo delayed gadolinium-enhanced cardiac magnetic resonance (LGE-CMR) imaging and ex vivo T1 mapping. Methods and results Ten pigs with established myocardial infarction (MI) underwent in vivo scar characterization using LGE-CMR imaging and high-density voltage mapping of both ventricles using a 3.5-mm tip catheter. Ex vivo post-contrast T1 mapping provided a high-resolution reference. Voltage maps were registered onto the left and right ventricular (LV and RV) endocardium, and epicardium of CMR-based geometries to compare voltage-derived scars with surface-projected 3D scars. Voltage-derived scar tissue of the LV endocardium and the epicardium resembled surface projections of 3D in vivo and ex vivo CMR-derived scars using 1-mm of surface projection distance. The thinner wall of the RV was especially sensitive to lower resolution in vivo LGE-CMR images, in which differences between normalized low bipolar voltage areas and CMR-derived scar areas did not decrease below a median of 8.84% [interquartile range (IQR) (3.58, 12.70%)]. Overall, voltage-derived scars and surface scar projections from in vivo LGE-CMR sequences showed larger normalized scar areas than high-resolution ex vivo images [12.87% (4.59, 27.15%), 18.51% (11.25, 24.61%), and 9.30% (3.84, 19.59%), respectively], despite having used optimized surface projection distances. Importantly, 43.02% (36.54, 48.72%) of voltage-derived scar areas from the LV endocardium were classified as non-enhanced healthy myocardium using ex vivo CMR imaging. Conclusion In vivo LGE-CMR sequences and high-density voltage mapping using a conventional linear catheter fail to provide accurate characterization of post-MI scar, limiting the specificity of voltage-based strategies and imaging-guided procedures.

Fully-Coupled Electromechanical Simulations of the LV Dog Anatomy Using HPC: Model Testing and Verification

Verification of electro-mechanic models of the heart require a good amount of reliable, high resolution, thorough in-vivo measurements. The detail of the mathematical models used to create simulations of the heart beat vary greatly. Generally, the objective of the simulation determines the modeling approach. However, it is important to exactly quantify the amount of error between the various approaches that can be used to simulate a heart beat by comparing them to ground truth data. The more detailed the model is, the more computing power it requires, we therefore employ a high-performance computing solver throughout this study. We aim to compare models to data measured experimentally to identify the effect of using a mathematical model of fibre orientation versus the measured fibre orientations using DT-MRI. We also use simultaneous endocardial stimuli vs an instantaneous myocardial stimulation to trigger the mechanic contraction. Our results show that synchronisation of the electrical and mechanical events in the heart beat are necessary to create a physiological timing of hemodynamic events. Synchronous activation of all of the myocardium provides an unrealistic timing of hemodynamic events in the cardiac cycle. Results also show the need of establishing a protocol to quantify the zero-pressure configuration of the left ventricular geometry to initiate the simulation protocol; however, the predicted zero-pressure configuration of the same geometry was different, depending on the origin of the fibre field employed.