Leopold Grinberg

Outflow Boundary Conditions for Arterial Networks with Multiple Outlets

Simulation of blood flow in three-dimensional geometrically complex arterial networks involves many inlets and outlets and requires large-scale parallel computing. It should be based on physiologically correct boundary conditions, which are accurate, robust, and simple to implement in the parallel framework. While a secondary closure problem can be solved to provide approximate outflow conditions, it is preferable, when possible, to impose the clinically measured flow rates. We have developed a new method to incorporate such measurements at multiple outlets, based on a time-dependent resistance boundary condition for the pressure in conjunction with a Neumann boundary condition for the velocity. Convergence of the numerical solution for the specified outlet flow rates is achieved very fast at a computational complexity comparable to the widely used Resistance or Windkessel boundary conditions. The method is verified using a patient-specific cranial vascular network involving 20 arteries and 10 outlets.

Modeling Blood Flow Circulation in Intracranial Arterial Networks: A Comparative 3D/1D Simulation Study

We compare results from numerical simulations of pulsatile blood flow in two patient-specific intracranial arterial networks using one-dimensional (1D) and three-dimensional (3D) models. Specifically, we focus on the pressure and flowrate distribution at different segments of the network computed by the two models. Results obtained with 1D and 3D models with rigid walls show good agreement in massflow distribution at tens of arterial junctions and also in pressure drop along the arteries. The 3D simulations with the rigid walls predict higher amplitude of the flowrate and pressure temporal oscillations than the 1D simulations with compliant walls at various segments even for small time-variations in the arterial cross-sectional areas. Sensitivity of the flow and pressure with respect to variation in the elasticity parameters is investigated with the 1D model.

Large-scale simulation of the human arterial tree.

1 Full‐scale simulations of the virtual physiological human (VPH) will require significant advances in modelling, multiscale mathematics, scientific computing and further advances in medical imaging. Herein, we review some of the main issues that need to be resolved in order to make three‐dimensional (3D) simulations of blood flow in the human arterial tree feasible in the near future. 2 A straightforward approach is computationally prohibitive even on the emerging petaflop supercomputers, so a three‐level hierarchical approach based on vessel size is required, consisting of: (i) a macrovascular network (MaN); (ii) a mesovascular network (MeN); and (iii) a microvascular network (MiN). We present recent simulations of MaN obtained by solving the 3D Navier–Stokes equations on arterial networks with tens of arteries and bifurcations and accounting for the neglected dynamics through proper boundary conditions. 3 A multiscale simulation coupling MaN–MeN–MiN and running on hundreds of thousands of processors on petaflop computers will require no more than a few CPU hours per cardiac cycle within the next 5 years. The rapidly growing capacity of supercomputing centres opens up the possibility of simulation studies of cardiovascular diseases, drug delivery, perfusion in the brain and other pathologies.

Simulation of the human intracranial arterial tree

High-resolution unsteady three-dimensional flow simulations in large intracranial arterial networks of a healthy subject and a patient with hydrocephalus have been performed. The large size of the computational domains requires the use of thousands of computer processors and solution of the flow equations with approximately one billion degrees of freedom. We have developed and implemented a two-level domain decomposition method, and a new type of outflow boundary condition to control flow rates at tens of terminal vessels of the arterial network. In this paper, we demonstrate the flow patterns in the normal and abnormal intracranial arterial networks using patient-specific data.

A new computational paradigm in multiscale simulations: application to brain blood flow

Interfacing atomistic-based with continuum-based simulation codes is now required in many multiscale physical and biological systems. We present the computational advances that have enabled the first multiscale simulation on 190,740 processors by coupling a high-order (spectral element) Navier-Stokes solver with a stochastic (coarse-grained) Molecular Dynamics solver based on Dissipative Particle Dynamics (DPD). The key contributions are proper interface conditions for overlapped domains, topology-aware communication, SIMDization, multiscale visualization and a new do- main partitioning for atomistic solvers. We study blood flow in a patient-specific cerebrovasculature with a brain aneurysm, and analyze the interaction of blood cells with the arterial walls endowed with a glycocalyx causing thrombus formation and eventual aneurysm rupture. The macro-scale dynamics (about 3 billion unknowns) are resolved by NεκTαr - a spectral element solver; the micro-scale flow and cell dynamics within the aneurysm are resolved by an in-house version of DPD-LAMMPS (for an equivalent of about 100 billions molecules).

Modeling of blood flow in arterial trees.

Advances in computational methods and medical imaging techniques have enabled accurate simulations of subject‐specific blood flows at the level of individual blood cell and in complex arterial networks. While in the past, we were limited to simulations with one arterial bifurcation, the current state‐of‐the‐art is simulations of arterial networks consisting of hundreds of arteries. In this paper, we review the advances in methods for vascular flow simulations in large arterial trees. We discuss alternative approaches and validity of various assumptions often made to simplify the modeling. To highlight the similarities and discrepancies of data computed with different models, computationally intensive three‐dimensional (3D) and inexpensive one‐dimensional (1D) flow simulations in very large arterial networks are employed. Finally, we discuss the possibilities, challenges, and limitations of the computational methods for predicting outcomes of therapeutic interventions for individual patients. Copyright

Numerical studies of the stochastic Korteweg-de Vries equation

We present numerical solutions of the stochastic Korteweg-de Vries equation for three cases corresponding to additive time-dependent noise, multiplicative space-dependent noise and a combination of the two. We employ polynomial chaos for discretization in random space, and discontinuous Galerkin and finite difference for discretization in physical space. The accuracy of the stochastic solutions is investigated by comparing the first two moments against analytical and Monte Carlo simulation results. Of particular interest is the interplay of spatial discretization error with the stochastic approximation error, which is examined for different orders of spatial and stochastic approximation.

A new domain decomposition method with overlapping patches for ultrascale simulations: Application to biological flows

We address the failure in scalability of large-scale parallel simulations that are based on (semi-)implicit time-stepping and hence on the solution of linear systems on thousands of processors. We develop a general algorithmic framework based on domain decomposition that removes the scalability limitations and leads to optimal allocation of available computational resources. It is a non-intrusive approach as it does not require modification of existing codes. Specifically, we present here a two-stage domain decomposition method for the Navier-Stokes equations that combines features of discontinuous and continuous Galerkin formulations. At the first stage the domain is subdivided into overlapping patches and within each patch a C^0 spectral element discretization (second stage) is employed. Solution within each patch is obtained separately by applying an efficient parallel solver. Proper inter-patch boundary conditions are developed to provide solution continuity, while a Multilevel Communicating Interface (MCI) is developed to provide efficient communication between the non-overlapping groups of processors of each patch. The overall strong scaling of the method depends on the number of patches and on the scalability of the standard solver within each patch. This dual path to scalability provides great flexibility in balancing accuracy with parallel efficiency. The accuracy of the method has been evaluated in solutions of steady and unsteady 3D flow problems including blood flow in the human intracranial arterial tree. Benchmarks on BlueGene/P, CRAY XT5 and Sun Constellation Linux Cluster have demonstrated good performance on up to 96,000 cores, solving up to 8.21B degrees of freedom in unsteady flow problem. The proposed method is general and can be potentially used with other discretization methods or in other applications.

Parallel performance of the coarse space linear vertex solver and low energy basis preconditioner for spectral/hp elements

The big bottleneck in scaling PDE-based codes to petaflop computing is scalability of effective preconditioners. We have developed and implemented an effective and scalable low energy basis preconditioner (LEBP) for elliptic solvers, leading to computational savings of an order of magnitude with respect to other preconditioners. The efficiency of LEBP relies on the implementation of parallel matrix-vector multiplication required by coarse solver to handle the h-scaling. We provide details on optimization, parallel performance and implementation of the coarse grain solver and show scalability of LEBP on the IBM Blue Gene and the Cray XT3.

Parallel multiscale simulations of a brain aneurysm

Cardiovascular pathologies, such as a brain aneurysm, are affected by the global blood circulation as well as by the local microrheology. Hence, developing computational models for such cases requires the coupling of disparate spatial and temporal scales often governed by diverse mathematical descriptions, e.g., by partial differential equations (continuum) and ordinary differential equations for discrete particles (atomistic). However, interfacing atomistic-based with continuum-based domain discretizations is a challenging problem that requires both mathematical and computational advances. We present here a hybrid methodology that enabled us to perform the first multi-scale simulations of platelet depositions on the wall of a brain aneurysm. The large scale flow features in the intracranial network are accurately resolved by using the high-order spectral element Navier-Stokes solver εκαr . The blood rheology inside the aneurysm is modeled using a coarse-grained stochastic molecular dynamics approach (the dissipative particle dynamics method) implemented in the parallel code LAMMPS. The continuum and atomistic domains overlap with interface conditions provided by effective forces computed adaptively to ensure continuity of states across the interface boundary. A two-way interaction is allowed with the time-evolving boundary of the (deposited) platelet clusters tracked by an immersed boundary method. The corresponding heterogeneous solvers ( εκαr and LAMMPS) are linked together by a computational multilevel message passing interface that facilitates modularity and high parallel efficiency. Results of multiscale simulations of clot formation inside the aneurysm in a patient-specific arterial tree are presented. We also discuss the computational challenges involved and present scalability results of our coupled solver on up to 300K computer processors. Validation of such coupled atomistic-continuum models is a main open issue that has to be addressed in future work.